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Preface & Abstract- Around the spring of 1974, my major professor, John P. Basart, came back from a 
Radio Astronomy conference where he had heard exciting news of a new approach to spectral estimation 
and computation. We discussed it at length and agreed that I should make it my PhD research topic. I had 
the background necessary to do research that was both theoretical and with practical applications. As I 
began the study of this ‘new’ topic, I realized that there was a general need to study a broader topic that 
included several methods that could be practical for sampled data from systems that inherently added 
noise to the measured signal. 

Throughout my research, I considered a complete development of the Maximum Entropy Method 
(MEM) my biggest challenge. I soon learned about the work of John Parker Burg, then at Stanford, and I 
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doing proprietary research for an oil exploration company. Due to many circumstances, including my 
decision to leave academia in 1976 for work on the new Global Positioning System, I never contacted 
him again. 

I was also unable to determine what papers he had published or the availability of his thesis until 
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of his early work but managed to find information that he and produced XX papers and XX presentations 
on MEM. They are included in the bibliography for completeness but I did not have access to them. 
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MEM research so don’t bother sending any more proposals! 
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are given and various spectral estimation methods are given. The theory of MEM is described in detail 
using approaches from estimation theory, communication theory, and statistics. The work was intended 
to give researchers the theory and practice of practical means of spectral estimation using 
communications or scientific data. The Maximum Entropy Method by John Parker Burg is explained 
from what was known in 1974-75. 
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PREFACE 


Although there are copious publications available that address the 
topic of spectral analysis in general and many that address time series 
data in particular, it is difficult to find information on how to apply 
the general theory to practical data processing. User oriented applica- 
tion techniques and computer programs are seldom published and canned 
routines ‘eave the researcher uneasy as to their biasing effect on the 
results. This research was done in an effort to provide a complete system 
of applied spectral analysis. Special attention was given to describing 
the effects of measurement and sampling error, observer bias, and 
interpretation of results. 

Much of the text is basically tutorial and, as such, is not original 
but the organization and presentation are unique. Except for Chapter XIX, 
the original contributions are discussed in the introduction. The author 
is especially happy with Chapter XIX because it represents several ideas 
on interpretation he believes to be unique. To the fullest extent possible, 
authors of works already published have been gratefully acknowledged by 
the many references. 

The author's major professor, John Basart, has contributed greatly to 
this work through his helpful technical discussions, objective criticism of 
important ideas, and diligent proofreading. His original interest in the 
Maximum Entropy Technique and conviction that it would provide an important 
dissertation topic are the embryos of this work. It could not have been 


completed without his continuing encouragement and patience. 


ix 


The author's very special thanks go to his wife, Kathy, for her con- 
stant encouragement that gave him the desire to finish a difficult task 
and to his daughters, Carla and Tracy, who spent many of their growing years 
with a "student" father. 

The author is indebted to Betty Carter for typing the manuscript and 
for her excellent formatting of many difficult equations. 

Readers interested in these topics are encouraged to contact the 
author concerning the ideas presented not only to discuss their merits but 


also to generate new and better approaches. 


Ames, Iowa Steve F. Russell 
January, 1978 


I, INTRODUCTION 


The analysis of empirical data for the purpose of determining funda- 
mental parameters for a physical system is a basic goal for much applied 
research. One of the most important characteristics to be determined is 
the frequency spectrum of the recorded data. Spectral analysis plays an 
important role in many scientific fields such as radio astronomy, meteo- 
rology, geophysics, mechanics, statistics, communication signal processing, 
econometrics, physics, and biology. Probably the most important research 
applications of spectral analysis are presently in the fields of geophysics 
and astronomy. The most advanced work on the spectral analysis of noisy 
empirical data is being published by geophysicists in connection with oil 
exploration geology. The search for new sources of oil in deeper domes 
and offshore has placed severe requirements on the available methods of 
data processing. To improve the spectral resolution of the measured spec- 
trum, these researchers have developed many sophisticated techniques with 
the most advanced being maximum entropy spectral analysis. 

Maximum entropy spectral analysis (MEM) has stirred the interest of 
radio astronomers because its claims for better spectral resolution with 
limited amounts of data creates a promise of better sky brightness maps for 
radio sources. However, both the theory and application of MEM are not well- 
developed and, consequently, this tool is not available to most researchers. 
Initially, the goal for this research was to develop the theory and prac- 
tical implementation of the maximm entropy spectral analysis method for 
estimating the power spectral density function when the autocorrelation 


data are incomplete. As the research progressed it became apparent that 


many problems and barriers existed between theory and application of spec- 
tral analysis. These problems were further investigated and it was con- 
cluded that a more general approach to spectral analysis should be taken. 
This approach led to a more user oriented thesis. Such topics as estima- 
tion theory, frequency domain analysis, system analysis techniques, sam- 
pling theory, and Fourier analysis were investigated. The selection of a 
method of spectral estimation as well as the mathematical behavior of che 
most common methods was developed. 

The author's original contributions are a collection of smaller topics 
scattered throughout the text. Chapter X, the Hilbert space vector for- 
milation of digital Fourier transform analysis, is independent work that 
constitutes the mathematical framework used to develop the spectral mixing 
formula discussed in Chapter XI. The discovery of a companion function to 
the time autocorrelation function is original work and is discussed in 
Chapter XV. The mathematical development of the calculus of variations 
technique used for this derivation was developed independently and is 
given in Chapter XXI, Part J. Much of the development of digital Fourier 
transform spectrai estimation given in Chapter XII is unique to this text. 
The author's approach is one of analysis and interpretation while most 
published material concentrates on the computational details of the fast 
Fourier transform (FFT). The chapter on methods of spectral analysis is 
original work designed to aid researchers in selecting appropriate esti- 
mation techniques and to heip in understanding the problems common to all 
spectral estimators. Although the topics of sampling theory presented in 


Chapter VI are not new, the presentation and derivations are specifically 


designed to relate the effects of sampling to spectral analysis. The dis- 
cussion of MEM is not original but the penne program and interpretation 
of results have not been previously published. 

Much work remains to be done in this research area. More theoretical 
study of MEM is needed to determine optimum series lengths and more effi- 
ciert methods of spectral calculation. Data reduction schemes and better 
interpretation of estimated spectra need further research work. Probably 
the most needed future research is on the problem of unequally spaced data 
samples and undersampled data. 

Most of the mathematics and terminology used in the text are those 
commonly used in current publications in the spectral analysis area. One 
of the author's more fundamental contributions has been to unite the ter- 
minology used by geophysicists, statisticians, and electrical engineers. to 
provide a common basis for commnication. Where more than one terminology 
exists for a given concept, every effort has been made to define and use 
the varied terminology. The term time function is used to denote a contin- 
uous function of time that can be completely described in mathematical 
terms. A time series is a sampled or discrete time function and may be 
ideal or empirical. The term data function is used by the author to denote 
either a time function or time series that is being numerically processed. 
The author defines collective parameters as those characteristics of a data 
function that are used to describe the process that generated the data. 

An asterisk is used to denote complex conjugation, a script F operator 


denotes Fourier transformation ( F{x(t) }=X(w)), and convolution is abbre- 


Vw 
+ 
=m 
on 
ct 
wv 


viated by the product; y(t) =x(t 


This research and resulting thesis are most directly suited for use by 
the researcher who mist du very serious spectral studies. The casual user 
is better served by applying readily available "canned" approaches to spec- 
tral analysis. This distinction and the possible alternatives available 
are discussed in Section D of Chapter IX. 

It will become apparent to the reader doing a detailed study of this 
work that the techniques of autoregressive spectral estimation, maximm 
entropy spectral estimation, and recursive digital filtering all have 
similar mathematical forms. They do share a common mathematical dna. 
theoretical basis but they produce different spectral estimates because 
they employ different criteria-of-goodness and numerical algorithms. These 
similarities were useful in helping to better define the properties of 
these estimators. 

It must be emphasized that this work has a wide range of applicability. 
It is useful for any empirical work involving the Fourier transform or the 
autocorrelation function. The variables of time and frequency are used for 
convenience but others such as space and momentum are equally valid. All 
the methods of analysis presented involve some type of model fitting. In 
estimation theory, a truly "parameter free" method is not possible. The 
asieceidn.oe a model is the most important task in spectral analysis. 
Chapter IX was written to help clarify the selection of a model and the 


computing method. 


II. TIME AND FREQUENCY CHARACTERIZATION OF TIME 
FUNCTIONS AND SERIES 

A time function or time series can always be specified by enumerating 
all possible function vaiues in the time domain. For the continuous time 
function, this is done by representing the function by a mathematical equa- 
tion. For the discrete time series, the entire description can be given by 
an autoregressive equation or difference equation. This complete descrip- 
tion often involves mathematical completeness in the sense that the math- 
ematical representation is valid over the entire time domain and is well 
behaved. In many scientific applications, this complete description is not 
necessary. Quite often it is sufficient to specify more collective param- 
eters such as average value, mean-square value, or power spectrum. Such 
collective descriptions often give valuable insight into the physical nature 
of the process which generated the observed data. For this reason it is 
desirable to discuss some of these collective characterizations. 

For theoretical purposes, a time function or time series can be con- 
Sidered as being a sample function of a real process. This process can 
generate a time function which has both periodic components and random 
components. Often a statistical description involving probability density 
functions and higher order moments is considered suitable. For most cir- 
cumstances, the observable in this system is some time function or series 
and it is the analysis of this time data that mist be used to study the 
process, The definition of many of the collective parameters of a real proc- 
ess involve the use of statistical definitions. To be able to obtain these 


same parameters from time data it is usually necessary to assume that the 


process is ergodic. With this assumption it is possible to avoid the use 
of statistical descriptions of the process and obtain the collective pa- 
rameters directly from the time data. Most of the theory that is presented 
in this dissertation will be concerned with time functions that are sample 
functions of real ergodic processes. The use of probability descriptions 
will usually be limited to those that are necessary for a better theoret- 
ical understanding. None of the actual signal processing techniques will 
involve the direct use of any probabilistic descriptions. 

In the time domain, a data function can be characterized by its time 
average value, mean-square value, and variance. These collective parameters 
can be related to the statistical description of the function and yet do not 
require any transformations or use of statistical parameters. The time do- 
main analysis of a data function can also involve the computation of the 
time autocorrelation function (Appendix I, Part E). The time autocorrela- 
tion function is a measure of how well the function correlates with itself 
in time and also can be used to determine the mean-square value of the func- 
tion and detect any periodic components of the process. 

In the frequency domain, a data function can be characterized by its 
amplitude spectrum or its power spectrum. The amplitude spectrum is ob- 
tained by taking the Fourier transform of the data function. The existence 
of an amplitude spectrum depends upon the existence of the Fourier trans- 
form. In many cases, the amplitude spectrum of a random process does not 
exist. The power spectral density function for a data function is obtained 
by taking the Fourier transform of the autocorrelation function. This def- 


inition is superior to the use of the amplitude spectrum because it elim- 


inates the problem of existence. The power spectrum of a data function is 
often the desired physical information that is needed to solve a problem or 
verify some scientific hypothesis. For discrete data the power spectrum 
can be determined from the discrete form of the autocorrelation function or 
by using the autoregressive power spectrum. 

In theoretical discussions of the characterization of data functions, 
it is possible to give a mathematical formulation which describes the func- 
tion over the entire domain of the independent variable. When the data 
function is obtained from a physical process, the mathematical description 
becomes only an estimate for the "true" function. The various collective 
parameters which are used to characterize the data function are derived 
with the infinite domain assumption. Measured data can be described by 
collective parameters only when mathematical estimates of the entire data 
function are assumed. For example, a finite time series can be represented 
by a discrete Fourier series or by an autoregressive series. Both of these 
representations assume that the data function can be defined for all time. 
In Fourier analysis this is referred to as the periodic extension assump- 
tion. 

Once a mathematical description of the data function is obtained, the 
various collective parameters such as average value, mean-square value and 
power spectrum can be calculated from the theory. As a consequence, these 
representations are only valid for the infinite domain assumption. It is 
important to realize that a concept such as power spectrum is totally de- 
pendent on the mathematical definition and is not related directly to a 


physicai concept. The mathematical definitions for the various collective 


parameters are given in Chapter IV. 


III. OBSERVABLES AND ESTIMATION THEORY 


The mathematical description of an area of physical theory such as 
quantum mechanics or spectral analysis often involves the use of transfor- 
mations into other domains. When working with this theory, it is important 
to realize that quantities defined in domains other than the time domain do 
not have a real physical significance and hence are not physical observ- 
ables. Quantities such as amplitude spectra or power spectra depend upon 
a mathematical definition for their existence. The definition is usually 
chosen so that it helps to give additional useful information about the 
physical process that is described by the function. As an example, for a 
periodic function, the amplitude spectrum obtained by taking the Fourier 
transform gives the complex amplitude coefficients of the Fourier series 
which describe the time behavior of the function. 

A time function can be defined for all time but a real observable can 
be defined only for a finite time. This causes problems with definitions 
which involve the use of infinite time functions. Transform domain func- 
tions such as amplitude and power spectra are defined only for functions 
which exist for all time. These definitions must be modified to accommodate 
the "finiteness" of a real observable. The modification is usually designed 
so that it gives a good approximation of the "true" value. Another approach 
is to use mathematical models to approximate the observed function with a 
mathematical function which is defined for all time. The study of how to 


formulate mathematical functions which describe data that is obtained 


empirically is called estimation theory. 


The simplest forms of estimation theory are concerned with the selec- 
tion of an approximating function to "fit" a set of empirical data and the 
selection of a criterion to judge the "goodness" of that fit. A function 
or computational scheme that is used to describe some property of the empir- 
ical data is referred to as an estimator. Estimators can be as simple as a 
single sine function or as complicated as a digital Fourier transform. The 
three estimation processes most commonly used are those of signal filtering 
or smoothing, prediction or extrapolation, and interpolation. These pro- 
cesses are typically used in conjunction with a desired mathematical process 
such as numerical intergration or Fourier transformation. The estimation 
processes of smoothing, extrapolation, and interpolation are all useful in 
spectral sisiyoie: 

Estimation theory is used to help solve the common problems of numer- 
ical integration, difference equations, and the representation of empirical 
data by exact functions. These methods are discussed in books on estima- 
tion theory and numerical analysis. A partial list of these methods would 
include for interpolation; Lagrange interpolation, Hermite interpolation, 
divided differences, Newton's forward and backward differences, Gaussian 
interpolation, Stirling's interpolation, and methods due to Bessel, 
Everett, and Steffenson, and for numerical integration; Riemann summation, 
Cote's formulas, Simpson's rule, Romberg's method, Filon's forma, Gauss-~ 
Sidel method, Gauss-Laguerre method, Gauss-Hermite method, Chebyshev's 
formulas, and aid-of differences, and finally for curve-fitting of empir- 
ical data; least-squares polynomial, orthogonal polynomials, trigonometric 


functions (Fourier series), exponential functions, Chebyshev polynomials, 


10 


continued fractions, and rational functions. 

Why are there so many approximation methods? The reasen is that es- 
timation theory cannot specify a unique method of estimation for every 
class of problem. All of the listed methods incorporate assumptions about 
the measured data that make that estimate optimum in some sense. For 
every criterion of goodness, a slightly different estimator can be deter- 
mined. 

Estimation methods often derive their name from the criterion used in 
their selection as "optimum". Such schemes as the method-of-least-squares, 
the method-of-moments, and the method-of-maximm-likelihood are often seen 
in the literature. Estimators can also be classified according to such 
characteristics as linearity, bias, sufficiency, efficiency, and asymptotic 
efficiency. 

The selection of an estimate as "best" is strictly a matter of user 
definition. Experience with the use of practical estimators has shown 
that certain criteria produce good results for a variety of tasks. One ée 
the most popular criterion-of-goodness is to minimize the least-square 
error between data values and estimated values. The sum given in Equation 
3-1 is often referred to as the total mean-square error or "residue" of 
the estimate: 

a 2 


1 > A 
mean-square error = N (x, - x) 
n=1 


(3-1) 


The data set is represented by the x values and the estimations by the 


A 
x values. 
n. 
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Estimation theory is an important part of the theory of spectral 
analysis of empirical data because of the finite observations involved. 
Estimation theory can be applied to truncated functions to provide the 
"best" possible spectral estimate for a given amount of data. Criteria of 
goodness for spectral estimators are often not well-defined and an attempt 
will be made to discuss this problem for each of the various estimation 
schemes. 

A rigorous background in estimation theory is not necessary for the 
study of spectral analysis but it does help to clarify the goals of the 
various schemes and puts the selection of a particular scheme on a ration- 
al basis. For the reader who is interested in a good introductory text on 


estimation theory, the book by Deutsch (1965) is recommended. 


12 


IV. TIME AND FREQUENCY DOMAIN ANALYSIS 


The time and frequency domain analysis of a time function or time 
series can be done on a theoretical basis or on an empirical basis. The 
theory for this analysis has been well worked out and is widely published 
in communication theory books and statistics books. The empirical analysis 
of a data function involves the use of estimation theory and criteria of 
goodness and is not well defined for the user. Empirical analysis also 
involves many subjective decisions and a priori assumptions about the 
process which generated the data. These factors make it easy for the 
observor to bias and influence the computed results. This has been espe- 
cially true in spectral analysis where the experimenter often must choose 
among several techniques which give dissimilar results. In this chapter 
we will attempt to summarize the theory and equations that are used to 
characterize a data function and make special emphasis on the differences 


between theoretical definitions and estimations. 
A. Theoretical Definitions 


A data function is completely described by an equation or recursive 
relation which specifies all values in the time domain. Although this com- 
plete description is adequate, it is often more meaningful to describe the 
data function by such collective parameters as average (mean or expected 
value) value, mean-square value, variance, autocorrelation function, am- 
plitude spectrum, and power spectrum. A discussion of the theoretical def- 


initions for these parameters will now be given. It will be assumed that 
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the data function is defined for all time and is well~behaved so that there 
are no mathematical problems with the definition. The theoretical defi- 
nitions will depend on a knowledge of the probability density function for 
the process and on the assumed ability to define the function over the 
entire domain of the independent variable. Most of the theory to be given 
can be applied to deterministic functions as well as random time functions. 
The tscueston will emphasize the random aspects of the data function but 


it will be assumed that the reader will see the analogy for deterministic 


functions. 


The expectation (or average) value for a random variable is defined 


as: 


ce a Gr (44.1) 
all x 


This theoretical definition depends on a complete knowledge of the prob- 
ability density function p(x) and the existence of the integral. 


For a periodic function, the average value for one Fourier period is 


computed from the integral: 
x(t) = 2 0 xceyat (48.2) 


The mean-square value for a random variable is defined as the second 


moment integral: 


E{x2} = { x*p(x)dx (44.3) 
all x 


For a periodic function, the mean-square value for one Fourier 


period is: 
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x(t) Z x7(t)dt (4A.4) 


The square root of this value is called the root-mean-square value of the 
eavieaie function. The mean-square value for a random variable or 
periodic time function is often called the "power" of the function because 
it is a measure of the second moment and is used to calculate the power 
in an electrical circuit. 

The variance of a random function is a measure of the variability of 
the values of the function. The variance is defined as: 

of = f Ge - Efe) pmax (44.5) 
all x 

The variance is often referred to as the "ac power" in the function be- 
cause it is a measure of the variability after removal of the average or 
"de" part. An equivalent definition for the periodic function could be 
developed but is rarely used. Quite often the average value is subtracted 
from the total function and a "zero average" function is defined. This is 
done explicitly in the definition of the variance. A good example of 
this approach is the use of a zero-mean autocorrelation to define a power 
spectrum. Blackman and Tukey (1958) use the term autocovariance for this 
zero-mean function. Other descriptive parameters such as higher order 
moments could be defined but these are not often used. This completes 
the discussion of time descriptions and we now go to the autocorrelation 
function. 

The time autocorrelation function for a data function is defined by 


the following multiplication and integration process: 


a 2/2 
R = lim= +)d 4A.6 
(7 = Lim 5 ie ae t)dt (44.6) 


The autocorrelation fimetion has two very important uses. First, it can 
be used to describe the "coherence" of a function because it is a measure 
of how well the data function correlates with itself after a shift in time. 
The process of shift-miltiply-average emphasizes periodic features in the 
data funccion and is useful for determining any periodic structure in a 
data function which would normally appear to be stochastic in nature. 
Second, it is used to define the power spectrum of a stochastic process. 
The Wiener-Khinechine relations (Appendix I, Part F) define the power 
spectrum of a random function as the Fourier transform of its autocorrela- 
tion function. 

We refer to the autocorrelation function as the representation of the 
time function in the "autocorrelation domain,'' The domain variable Tt is 
called the lag time or simply the lag. The lag time is the amount of shift 
between the two functions, The autocorrelation function cannot be used to 
derive a unique time function because of the integration process. This 
means that the autocorrelation function is not unique with respect to the 
time function. For any given autocorrelation function, an infinite number 
of time functions could have been used to derive it. 

The frequency domain descriptions for a data function are limited to 
the amplitude spectral function, the power spectral density function, and 
the autoregressive spectral function. Other spectral representations such 
as uSing a filter transfer function are valuable for special applications 


. but are really just special cases of the three general methods. 
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The amplitude spectral function is defined as the Fourier transform 


of the data function: 
a - jut 
X(u) = f x(t)e Ie at (4A.7) 
—O 


The amplitude spectrum will exist only when the Fourier transform exists. 
It is most applicable for periodic functions and single pulses in the time 
domain. It also forms the basis for discrete Fourier transform analysis of 
sampled data functions where the function is assumed periodic beyond the 
sampling interval. 

The power spectral density function is defined as the Fourier trans- 
form of the autocorrelation function. These two functions form a Fourier 


transform pair as given by the Wiener-Khinchine relations: 


to 2 
S (uw) = [ BCne Mar (44.8) 
+" : 
RD = 35 [S, (eau (44.9) 


In theory, a knowledge of either function implies a complete knowledge of 
the other. In empirical analysis, this assumption is rarely true because 
of finite data. Consequently, the various estimators and algorithms will 
have an important effect on computed results. It is the development of 

the theory describing these various estimators that is the primary goal of 


this research. 
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B. Estimations 


When working with empirical data, the data function is only an esti- 
mate of the "true" function and all collective parameters that are calcu- 
lated from the empirical data set are estimated parameters. If the data 
set is large and the estimator is well chosen, the estimated parameters 
are very close to the true parameters. For theoretical reasons, and also 
for asthetic reasons, most estimators are chosen so that they are unbiased 
and converge to the true values as the amount of data becomes infinite. 
There are important cases where the estimate that is .sed does not converge 
to the theoretical value but still gives an acceptable estimate. This oc- 
curs often in commonly used techniques of spectral analysis. The estima- 
tors that will now be presented are widely used in the analysis of empiri- 
cal data. 

The average value for a time series or set of random variable values 
is estimated by the sum 


1 8 
en Ce 
HN kaka 
z 
1 & 


(4B.1) 


where the Pp, are the discrete probabilities associated with each data 
sample x It is commonly assumed that all of the samples, x,» are equally 
likely which means that the P, are all equal. For N equally likely random 


samples of x, the average value is the simple arithematic average given by 


(48.2) 
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For a periodic function, the average is estimated by the integral 


x(t) = x(t)dt . (4B.3) 
2 


where T is called the period of the average. Generally this period is not 
equal to the Fourier period because the Fourier period is unknown. The 
Riemann sum approximation for this integral is given by Equation 4B.2. 

The mean-square value for a discrete random variable is estimated by: 


2 1 82 
oes — en (4B.4) 
=1 k 


When all N samples are equally likely, the mean-square value reduces to: 


2 


8 2 (43.5) 


i 
N 
For a periodic function, the mean-square value is estimated by 
+T/2 


2 1 2 
x(t) == x" (t)dt (4B.6) 
z so 


where, again, T is called the period of the average (or estimate). The 
Riemann sum approximation for this integral is given by Equation 4B.5. 
The variance of a random function is estimated by using a weighted 


average summation approximation for Equation 44.5. This estimate is: 


a = (x-x)? = x* - (x)? (4B.7) 


When all samples are equally likely, (4B.7) is estimated by the sum: 


a2 _ N 


Be TD (45.8) 
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‘The estimation procedures used for autocorrelation functions, auto- 
covariance functions, amplitude spectra, power spectra, and autoregressive 
spectra are the most important aspects of spectral analysis and are the 
main themes of this dissertation. The procedure for selecting a particular 
estimation method and its application to empirical data will be discussed 
in Chapter IX. The details of the estimation algorithms will be presented 


in the appropriate later chapters. 
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V. SYSTEM ANALYSIS TECHNIQUES 


There are several important system theorems and mathematical tools 
used in spectral analysis. Many of these are associated with system 
analysis in electrical engineering and are also broadly applicable to many 
other physical systems. For completeness, and to familiarize the reader 
with many of the concepts to be used later, these analysis techniques will 


now be discussed. 


A. Convolution Theorem 


The time output response, y(t), of a physical system can be related 
to the time input, x(t), by convolving the system impulse response, h(t), 


with the input. A simple system is shown in Figure 54-1. For a linear, 


x(t) h(t) y(t) 


Figure 5A-1. A Simple Operating System for Illustrating Convolution 


time-invariant system, the convolution integral relating the input and out- 
put is written as: 
+o 
y(t) = [ h(t - 1)x(1)dt (5A.1) 
-O 


or in shorthand notation: 


y(t) = h(t) * x(t) (54.2) 
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All functions are assumed to be defined for all time and h(t-fT) is the 
impulse response of the system occurring at delay time, t=. 

The impulse response of a system is defined as the system output 
response when the input is a unit impulse (Dirac delta function). This 
can be shown by using the convolution integral of Equation 5A.1 and the 
sifting property of the Dirac delta function. 

The impulse response of a system can also be obtained by first com- 
puting the system transfer function, H(w), and taking the inverse Fourier 
transform. It is important to realize that H(uw) and h(t) form a Fourier 


transform pair: = 


to ud 
(Cu) = [ n(t)e Jae (5A.3) 
+00 ? 
h(t) = - ae H(ue 2 “ay (5A.4) 


This relationship is frequently used in electrical engineering to help 
analyze and synthesize electrical networks and systems. It is often easier 
to synthesize an electrical network in the frequency domain and then use 
the inverse Fourier transform to determine its time response. These equa- 
tions all assume infinite time functions and do not take into effect the 
requirement of eansareey in the system. 

For h(t) to be causal (no output prior to there being an input), the 
impulse response must be zero prior to zero time; h(t) =0, «<0. Also, if 
the system is to be stable, the impulse response must be absolutely in- 
tegrable (the response must die out as time goes on). The true limits of 


the convolution integral are actually the "overlap" of x(t) and h(t- 17). 
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Only for the theoretical case where both exist for all delay time, t, do 
the limits become infinite. At a specific observation time, t=t), the 
output will have a value corresponding to the value of the convolution 
integral: 

b 

y(t,) = f h(t, - Dx(a)dr (5A.5) 

a 

A typical graphical interpretation of this convolution integral is shown 


in Figure 5A-2. 


overlap 


Figure 5A-2. A Typical Graphical Interpretation of Convolution 
Showing Limits of Integration 


For physically realizable systems with causal impulse response, the 


convolution integral is usually written as (Schwartz, 1970, p. 87): 
t 
y(t) = f h(t - t)x(1)dt (5A.6) 
a) 


For both x(t) and h(t) causal, the convolution integral relating the 


system output with its causal input and causal impulse response becomes: 


y(t) = Since - 1)x(T)dt (5A.7) 
0 
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This representation of the system response most closely represents the 
true circumstances when a signal x(t) is applied to a system h(t). The 
system output, y(t), can be interpreted as a superposition of past values 
of x(t) weighted by. the system impulse response, h(t). A widely used 
graphical peeineced ek for the esaveletion of two causal functions is 
given in Figure 5A-3. The impulse response h(t- 1) represents the true 
system response after it has been "folded over" and extended into past 
time. The system response, y(t), at observation time t, is the input 
function, x(T), from tr=0 to tT=t weighted by the "folded over" impulse 
response, h(t - 1). 

The convolution integral of Equation 54.7 is used in linear systems 
analysis to develop the Laplace transform method of solving system re- 
sponses (Cheng, 1959, p 227). If x(t) and h(t) are both Laplace trans- 
formable, the Laplace transform of the output is equal to the product of 


the Laplace transforms of x(t) and h(t): 
Y(s) = X(s)H(s) (5A.8) 


This important result is called the convolution theorem. 

The convolution of finite time signals or periodic time functions 
with a causal system can be analyzed using the concepts of the amplitude 
transform, both Fourier and Laplace, but the analysis of the response of 
a system to a random time function must be done using autocorrelation 


functions and power spectra, This theory will now be summarized. 
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x(t) 


—— 
0 t 


a) Causal input function 


h(t) 


b) Causal filter response 


past future 


_— 


present 


¢) Convolution for causal functions 


Figure 5A-3. Graphical Interpretation for the Convolution of 
Two Causal Functions 
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B. System Response to Random Time Inputs 


~e 


A linear, time-invariant system, h(t), operating on a sample time 
function of a random process, x(t), defines a new sample function, y(t), 
that is related to the "oid" function and the system impulse response by 
the following convolution integral: 

+o 

y(t) = f h(u)x(t - u)du (5B. 1) 

—-@ 
This integrai is equivalent to that given in Equation 5A.1 and we see that 
the concept of convolution is valid even for random inputs. This alternate 
form of the convolution integral was chosen to make the following deriva- 
tions simpler. This result is very important because it will facilitate 
the derivation of important relationships between the input and output 
sample functions. 

If we assume the input process is both stationary and eiendies the 
time autocorrelation function can be used to determine the power spectra 
of input and output. The time autocorrelation function for the output is 
defined as: 

1 47/2 
R (+) = lim rp y(t)y(t+ t)dt (5B.2) 
y T7@ ~ =T/2 
The output at time t and at time (t+) can be written in terms of the 


convolution integral of Equation 5B.1 as 


+o 
y(t) = f h(u)x(t -u)du (5B.3) 


jo 
y(t+7) = f h(v)x(t+ 7- v)dv (5B.4) 


=o 
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where u and v are dummy variables of integration. If these relationships 
are substituted into Equation 5B.2, and the time average of the product 
x(t -u)x(t+t-v) is identified as the autocorrelation function of the in- 
put with argument (t-v+u), the autocorrelation of the output can be 
written in terms of the system impulse response and the autocorrelation of 
the input as (Carlson, 1968, p. 80): 
+o +o 

R(7) = Vy J BeedRCw)R, Cr - vt uavda (5B.5) 
Equation 5B.5 is an important intermediate result because it expresses the 
relationship between the input and output autocorrelation functions. [It 
can be thought of as the autocorrelation domain equivalent of the time 
domain convolution integral of Equation 5B.1. 

To obtain the power spectral density function for both the input and 
output processes, we take the Fourier transform of both sides of Equation 
5B.5. The resulting triple integral can be separated by rearranging the 
exponential of the transform to give an argument equal to the argument of 


the input autocorrelation function (Carlson, 1968, p. 81). This gives: 


+ 7 +<0 40 ; 
FRC} =f h(uje ts au { hive Jay i) R(t -vtu)e jw(t vtu),. 
~ 7 ~ (5B .6) 


The last integral is the Fourier transform of the input while the first two 
are the Fourier and inverse Fourier transforms of the impulse response. 


These transforms can be written in abbreviated form as: 


FRO) = Fh) 1 Flaw) JF RO} (5B.7) 


Using the Wiener-Khinchine relations and recognizing that the product of 
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the two transforms of h(t) is equal to the absolute value squared of the 
Fourier transform, we obtain the important relationship between input and 


output power spectra: 
2 
s,(w) = lHCw) |“s_ Cu) (5B.8) 


Equation 53.8 is widely used in the analysis of linear systems and their 
response to random inputs (Thomas, 1969, p. 145). It clearly illustrates 
the effect of a linear filter on the spectrum of an input function. It is 
also used to determine filter shape for optimum design and spectral 
shaping. This relationship is also used in prewhitening and in the design 


of a linear predictive filter as needed in maximum entropy spectral 


analysis. 
C. Digital Convolution 


Digital convolution can be performed with two discrete functions by 
adapting the convolution integral for causal functions as given in Equa- 
tion 5A.7. Since digital operations involve finite input data sets and 
digital filters of finite length, the convolution and filtering operations 
can always be thought of as involving causal functions. Approximating the 
continuous convolution integral with a Riemann sum gives 

t 


y, = At Dh, x (5C.1) 


where: 


t = the output or observation time index. 


At = the sampling interval. 
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The correspondence between discrete and continuous variables is given 


below: 


y(t) : y(tat) = y, 
h(t-71) : h(tat-kAt) = hy (5C.2) 


x(t) : x(KAT) = x, 


An example for a digital filter of length five and a time series of length 
six will now be given. Figure 5C-1 illustrates a graphical interpretation 


of digital convolution. The outputs for the example are: 


Yo = At(h)x,) 


¥, = At(h, Xo +h o*D 

Yo = At(h, x 9 t Byxy + Box,) 

ya At(hy x p Thox, +h) x, + hox 3) 

¥, = At(h,x, 9 thax, +h, x 2 thx, +h Ky, 
yom At(h,x, +h Xo + hox3 +h x ge 


¥,= At(h,x 9 thy eemenee se 6) 


¥z = At(h,x., +h,x, +hyx 5 + hyX6) 


vg At(h,x, +h,x, +hyx¢) 
vg" At(h,; x 5 + hX¢) 
Yyo = At(h, x6) 


These outputs represent the numerical computation of the digital outputs 

for observation times starting at zero and ending at 10At. The outputs Yo 
through ¥3 represent the transient response of the filter to the input, Yy, 
through V6 represent the steady-state output, and Y> through Y10 represent 


the "ringing down" of the filter. 
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Figure 5C-1. Graphical Interpretation for the Digital Convolution 
of a Time Series and a Digital Filter Showing the 
Output Conditions at ¥3 
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Digital convolution or filtering can also be thought of as the 
weighted superposition of past values of x(t). As time goes forward, the 
past values are represented by smaller values of t. If this weighting is 
represented by coefficients Ay2 Ay» Aye +++ ays the convolution can be 


written as: 


t 
Y_ = AE Fe (5¢.3) 


The digital convolution concept does not depend upon having a filter 
with as many coefficients, a,, as there are data values, X5° As a general 
rule, the filter is of length M and the time series is of length N. It is 
desirable to have N much larger than M so that filter transient response 
will be minimized. 

The most commonly used formulation for digital convolution is given 
by Equation 5C.4: 

M 
wae at oe ek (5C.4) 


The ranges on the indices are 


0,1, 2,3, 4, ... (N+M) 


ind 
it 


roy 
l 


= 0,1,2,3,4, ...M 


and the values for x with indices less than zero are assumed to be zero 
(causality). The response of the filter and the resulting output can be 


separated into three distinct observation time intervals: 
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1. ve for Ost<(M-1) is the transient response of the filter near 
the starting time. 

2. ve for MSt<SN is the steady-state response. 

3. ye for (N+1) <t<(N+M) is the "ringing down" or decay response 


after removal of the input. 


Digital filtering and convolution can also be formulated with matrices. 


This procedure is straight forward if one uses Equation 5C.4. 
D. Digital Deconvolution 


A general method for solving the convolution integral and obtaining 
a unique solution does not exist. In special cases, the convolution 
integral can be solved but the required boundary conditions and class of 
functions are such that these special circumstances do not apply to most 
practical cases. For discrete data and digital filters, decomvonicion is 
really a matter of definition and is an "unraveling" of the weighting 
applied by the filter coefficients. For most cases, digital deconvolution 
can be performed using the folllowiae method. 

Digital convolution was represented by Equation 5C.1. If the y's are 
measured and the h's are known, it is possible to use the set of (N+M+4+1) 
equations defined by (5C.1) to solve for the x's. This can be done re- 


cursively by solving the following system of equations: 


1 
x 


zy Nel 
om Se a 5D.1 
n Ata, Yn a, k=0 n-k*k ¢ ) 


These recursive equations can also be expressed exclusively in terms 


of the y's. When this is done, the resulting set of equations can be 
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thought of as a deconvolution filter with weights, b,- The "deconvolution" 


using this special filter is obtained from: 
(5D.2) 


The b's or filter weights are obtained from the a's of the original filter 


by using the following recursive formulas: 


o 
H 
| 

o 


1” 0 (5D.3) 


(n=2, 3, ...M) 
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VI. SAMPLING THEORY AND SAMPLED DATA FUNCTIONS 


In the usual sense, sampling theory is applied to sampled data systems 
in communication theory. In this application the user must consider such 
aspects as digital data transmission and reconstruction. The parameters of 
the system are usually under the control of the designer and can be opti- 
mized to suit a particular need. The signal is usually sampled at a rate 
well above the Nyquist rate and is sampled long enough to give any desired 
accuracy for the reconstruction. 

Sampling theory as applied to spectral analysis must be approached 
from a somewhat different viewpoint. First of all, the parameters of the 
transmitting system are not under the control of the experimenter. The | 
“received signal" is usually a data function generated by some physical 
process that is being investigated. Since the nature of the received sig- 
nai is being investigated for its unknown spectral properties, it is dif- 
ficult to determine what sampling rate is necessary. Also, the amount of 
data needed to resolve an unknown spectrum cannot be determined in advance. 
The experimenter must solve this problem in an iterative manner by making 
reasonable assumptions about the spectrum to be measured. The sampling 
technique can then be studied to see if these assumptions are valid. A 
widely used technique for spectral analysis of an unknown spectrum involves 
the use of prefiltering to limit the spectral input to the sampler. This 
method is most successful if the sampling system can be adjusted to produce 
a range of filtered spectrums and sampling rates. As can be seen from the 
theory that will be discussed, the prefiltering method will produce an ac- 


curate estimate of the input spectrum that is within the filter passband. 


A time function or time series generated by a real physical process 
can be generally Sesticeaes in the frequency domain by a spectrum con- 
taining both continuous and line components. Because of the "uncertainty 
principle" between time and frequency, this function is theoretically 
limited in spectral bandwidth by the physical process of measurement. 
Drawing on an analogy from quantum mechanics, the process of observation 
will introduce a perturbation which limits the observed spectral bandwidth. 
Even if the process has an infinite bandwidth, any measurement of a data 
function would involve the convolution of that function with a "filter" 
inside the measuring system. 

Because of this effect, every data function or set of empirical data 
will be bandlimited. To assure the user that the signal input to the 
sampler is bandlimited, it is wise to use the method of prefiltering. 

The method of data sampling can have a profound effect on an empirically 
derived power spectrum. The conditions of sampling such as sampling rate 
and number of samples are important parameters to be analyzed. The sam- 
pling technique is such a fundamental part of the measurement of a power 
spectrum that it is difficult to separate its effects from the actual 
power spectrum estimation technique. 

The basic concepts of sampling theory as applied to spectral analysis 
will be discussed in this chapter. It will not be possible to discuss all 
aspects of this broad topic but the most important ideas will be covered. 
Specific applications of sampling theory to the various aspects of spectral 


estimation will be presented in appropriate later chapters. 
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A. Sampling Theory - Basic Definitions 


The important parameters of a sampling system are the spectral band- 
width of the data function to be sampled, the sampling rate, the pulse 
shape of the sampling function, the output filtering for the sampler, and 
the amount of data measured. Figure 6B-1 shows a block diagram of a sam- 
pling system that will be used for the system analysis of the sampler. It 
is assumed that prefiltering has already been applied to the data function 
and that post-sampling reconstruction or filtering will be applied at a 
later stage. For purposes of discussion it will be assumed that the proc- 
ess being examized produces a continuous time function. 

The time function to be measured, x(t), is called the sampled function. 
It is assumed to be continuous, bandlimited, and defined for all time. 

The spectral bandwidth of x(t) will be assumed to be limited to 2u,- The 
output of the sampler is denoted by y(t) and is obtained by multiplying the 
sampled function, x(t), by the sampling function, m(t). m(t) is charac- 
terized by a repetitive pulse shape, p(t), and is a periodic function that 
can be represented by a Fourier series. The sampling system can also be 
modeled by the convolution of a Dirac comb function and a single pulse, 
p(t), which represents the basic pulse shape of the sampler. The system is 
modeled with system blocks such as multiplication and convolution so that 
the system analysis techniques given in Chapter V can be used in the de- 
rivations. 

The important relationship between the sampling rate (or frequency) 
and the Nyquist sampling rate (or frequency) can be discussed with the help 


of Figure 64-1. The Nyquist frequency depends upon the spectral content of 
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c) Sampling at less than the Nyquist frequency (undersampling). 


Figure 6A-1. Output Spectra for an Ideal Sampler Showing the Effects of 
Sampling Rate and Aliasing 
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the signal being sampled and is defined to be twice the highest frequency 
for which a nonzero spectral component of x(t) exists (Schwartz, 1970, 
p. 119). For the present discussion, the spectral components in x(t) will 
be limited to frequencies less than tb so the Nyquist frequency is fy = 24,- 
The time interval between samples, i, > is called the Nyquist sampling 
interval. 

The sampler is characterized by the sampling frequency, sampling in- 
terval, and folding frequency. The time interval between samples (sampling 
interval) is T. and the sampling frequency is defined as the reciprocal of 


the sampling interval, f=. In this dissertation, the folding frequency 
s 


will be defined to be one-half of the sampling frequency, £.=5f,: The 
folding frequency is a characteristic of the sampler and is the highest 
frequency that can be sampled without aliasing effects. This definition 
and concept is important to the discussion of the theory of sampling. 

Other authors appear to define it in a different manner (Blackman and Tukey, 
1958, p. 32). 

If a signal is sampled at exactly the Nyquist rate, the Nyquist fre- 
quency and the sampling frequency will be equal and the folding frequency 
and the frequency of highest spectral content will be equal. This condi- 
tion is often used as an example in discussions of sampling but does not 
illustrate the important differences between the characteristics of the 
sampled signal and the characteristics of the sampling device. 

The sampling rate can have a drastic effect on the spectral output of 
the sampler. The sampling rate mst be greater than the Nyquist rate 


or serious spectral distortion will occur. From the viewpoint of the 


sampling device, this is equivalent to saying that the input signal cannot 
have any spectral components above the folding frequency or aliasing will 
occur. This phenomenon, which is called aliasing, is illustrated in 
Figure 6A-1. The input spectrum to the sampler is assumed to be triangu- 
lar. Figure 6A-la shows the spectrum of the output, y(t), when the sam- 
pling frequency is greater than the Nyquist frequency of the input. Figure 
6A-1b shows the output spectrum when the sampling frequency is equal to the 
Nyquist frequency and Figure 6A-1c shows the aliasing effect when the sam- 
pling frequency is less than the Nyquist frequency. Sampling at a rate 
that is less than the Nyquist sampling rate causes the output spectra to 
overlap. This overlap or "folding back" of the spectra creates erroneous 
Spectral components or "aliasing" of the true spectrum. It is important to 
note that not all of the spectral information is rendered useless by alias- 
ing. The spectral components below a frequency of (f,- » remain unaf- 


fected. 
B. Sampling Theory - The System Equations 


Figures 6B-2 and 6B-3 represent the sampling system in both the time 
and frequency domains. Both representations are used because both provide 
useful system information. The solution for the output spectrum of the 
sampler will be obtained by solving the convolution interals by means of 
the Fourier transform. 

The sampling function, m(t), can be represented as the convolution of 


the pulse shape of the sampler with a Dirac comb function (See Equation 


6C.2): 
+o 
m(t) = f p(T)Comb(t - T)dt (6B.1) 
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The sampling function is even and periodic so it can be represented by an 


even Fourier series 


(tc) -4+c¢ +c cos (2M tyes T (6B.2) 
= BG ey T. = p(t +n.) - 


n=-0 
where C,/2 is the average value and qT. is the sampling rate. 
The output of the sampler is obtained by multiplying the sampling 
function, m(t), and the sampled function, x(t). The resulting output can 


be’ written in two ways: 


+2 
y(t) = x(t)m(t) = x(t) F p(t - nT.) (6B.3) 


n=-o 


In the frequency domain, the convolution integral representing m(t) is re- 


placed by multiplication of the Fourier transforms of p(t) and Comb(t) 
M(u) = P(w)Comb(w) (6B.4) 
where: 


Mu) =F {m¢t) } 
P(w) =F {pce)} 
Comb(u) = $ {Comb(t) } 


Also in the frequency domain, the mltiplication of x(t) and m(t) is 


replaced by the convolution of their respective Fourier transforms: 
1 +o 
YCw) = 5a f XCAMCwe AAA (6B.5) 
-a 


These system equations describe the operation of the sampler and will now 
be used to derive equations for the output frequency spectrum and the re- 


constructed signal. 
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C. Output Frequency Spectrum 


The output frequency spectrum of the sampler can be obtained by 
making use of the special properties of the Dirac comb function or by 
expressing the sampling function as a Buus series. Each method will 
now be briefly presented. 

The reason for expressing the periodic sampling function, m(t), as the 
convolution of a single pulse, p(t), and a Dirac comb function is that the 
spectrum of m(t) can be expressed as the product of the spectrum of p(t) 
and the spectrum of Comb(t). Since the spectrum of a Dirac comb function 
is another Dirac comb function, the convolution integral of Equation 6B.5 
is easily solved by using the sifting property of delta functions. First, 


the amplitude spectral density function for m(t) is: 
M(W) = PC w)Comb( (6C.1) 
The Dirac comb function is written in the time domain as: 
fo 


Comb(t) = 5 6(t - nT.) (6C.2) 


The Fourier transform of the Dirac comb function is itself a Dirac comb 


function (Reference Data for Radio Engineers, 1968, p. 42-2): 


F (comb(t)} = comb(w = 2 a, urn FD (6C.3) 
Steere s 


Since the output spectral fimction is obtained from the convolution 


of M(w) and X(w), we can write Y(w) as 


1 too 
YCu) = a7 X(A)PC we A)Comb(w- A)dA (60.4) 


ied? °) 
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where M(w- A) =P(w-A)Comb(w- A). Explicitly writing out the series for 
the Dirac comb function and replacing zs with W, » the convolution integral 
s 


of Equation 6C.4 becomes: 
lL pe t= 
¥(w) = 55 J kO)p(w Adu, 6(w- A- nu) 1dr (6C. 5) 


Equation 6C.5 is reduced by cancelling common factors and inter- 
changing summation and integration to give: 
+o 
DY 


+o 
x(a) = 2 Ef KCADP(w- A)E(w= A- MYL)AA (6C.6) 
s “2 


N=-0 


The integral can be eliminated by making use of the sifting property 
(Appendix II, Part E) of the Dirac delta function. Using the sifting 


property for Equation 6C.6, the convolution integral becomes 
+o 
J XCAYPCw- 0) 6(A = wt nw dd = X(w- ny, )P(nw,) (6C.7) 
“co 

and the output spectrum for the sampler can be written as: 


+o 
y P(nw,)X(w- nw, ) (6C.8) 


=—o 


Y¥(w) = t 
The term P(nw,) represents the Fourier transform of the sampling pulse, 

p(t), as it is periodically sampled at values nw, in the frequency domain. 
Since this amplitude function is a constant with respect to the frequency 


variable, w, and is only dependent on the sampling frequency W, » it can be 


replaced by a real coefficient defined by: 


Pn = P(nw,) (6C.9) 
s 


The output spectrum can now be represented by the infinite series 


+c0 
YCw) = ote PXCw- nw.) (6C.10) 


with constant coefficients P, and a periodic repetition of the amplitude 
spectrum of the sampled data function, x(t). Before interpreting the 
meaning of the output spectrum, the alternate derivation using the Fourier 
series representation for the sampling function, m(t), will be given. 

The sampling function is represented by the Fourier series of Equation 
6B.2. The Fourier transform of this series given by 

co 
M(w) = mC, 6(w) + EG, wLé (nu, - w) + 6(nw, + w)] (6C.11) 

is used in the convolution integral for the output spectrum given by Equa- 
tion 68.5. The Fourier transform for m(t) is substituted into (6B.5) and 
integration and summation are interchanged to give the integrals: 


1 +o 
Y(w) = 5 C, J XCADECA- war 


-~ 
1 <) +e 
+> rc [x(a [6(a- wenw,) + 6(A- we nw) 1dr (6C.12) 
2 y=1 2 ee s Ss 
The sifting property of the Dirac delta functions is used to eliminate the 
integrals and give an output spectrum of: 
1 L ee 
YCw) = 2 CXC wu) + C_X(w- nw.) +5 one nw,) (6C.13) 
n=-o p 
The first term represents the original spectral function centered about 
zero frequency and amplitude scaled by a factor of C,/2- The second term 
is the series of all the negative frequency spectral terms and the third 
series represents all of the positive spectral terms. If these series are 


identified term by term with the previous result given by Equation 6C.10 we 
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see that the coefficients are related as: 


-2 
Ch = = P(nw,) (6C.14) 


'g 
i) 


Cc /2 

n 
The function P(nw,) is even so the coefficients, P, oF C. form an even 
array (P, ie ee c_. . 

A simple example will now be given to illustrate the application of 
the derived equations. Consider a sampler whose sampling function consists 
of a pulse train of narrow rectangular pulses as illustrated by Figure 6C-1. 
The amplitude spectrum of x(t) will assume the arbitrary but bandlimited 


form shown in the figure. The amplitude spectrum of a single pulse of m(t) 


is given by the Fourier transform of a narrow rectangular pulse: 


AT. 
+ . Ar 
Fiece)}= f 7 aciMae = a sin(= «) (6¢.15) 
; AT. 


This equation can be written with a sinc function: 
aT. 
sin > w) 


iad) 
2 W 


PCy) = AAT. (60.16) 
The pulse height of the sampler, A, is called the gain of the sampler and 
AT. is the pulse width. The product AAT. is the area under the rectangular 
pulse. For purposes of analysis, the pulse area is often set equal to 
unity. The spectrum of p(t}, sampled at intervals corresponding to nw, , 


is used to obtain the coefficients, Pac These coefficients are calculated 
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from: 


AT 
sin(—~ nw_) 
P(nu.) = AAT. iS (6C.17) 


The output spectrum for the rectangular sampler can be calculated from 
Equation 6C.10 and the corresponding amplitude coefficients from (6C.17) 
and (6C.9). The output spectrum for the rectangular pulse example is shown 
in Figure 6C-2. 

As the pulse width of the rectangular pulse is made narrower, and the 
pulse height increased so that the pulse area remains constant, the rec- 
tangle function approaches a delta function. If the sampling function, 
m(t), is a Dirac comb function, the amplitude spectrum of a single delta 


function is a constant and the output spectrum simply becomes 


aT. +e 
YCu) =A r by X(w- nw.) (6C.18) 
s n=-< 


because all the PL, coefficients are equal. The original spectrum is repro- 
At 


duced periodically with an amplitude scaling factor of A — This type of 
s 


sampler is called an ideal sampler. The sampler that directly multiplies 
the incoming signal by a rectangular pulse shape is called a “natural" 
sampler, an exact-scanning sampler, or a mltiplying sampler. When the 
data function or input signal is sampled with a sample-and-hold circuit, it 
is referred to as square-topped sampling. A similar analysis of the square- 


topped sampler yields an output spectrum which is given by: 


1 = : 
YCw) = r ee XCw- nw.) (6C.19) 
s =o 
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x(t) 


Figure 6C-l. Waveforms and Input Spectrum for Rectangular 
Sampling Pulses in a Multiplying Sampler 


Figure 6C-2, Output Amplitude Spectrum for a Multiplying Sampler 
with a Rectangular Multiplying Function 
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This spectrum differs from that of the exact-scanning spectrum in that the 
Fourier transform of the sampling pulse is now an explicit function of the 
frequency, w. This can be interpreted as a weighting function on the 
periodic spectrum that tends to lowpass filter the spectrum that would 

be produced by an ideal sampler. If m(t) is a rectangie function, the 
weighting function will be a sine function. This effect may or may not be 
desired and certainly must be taken into account if a sample-and-hold 
circuit is used. 

From the general theory of sampling that has been presented it is 
possible to summarize some of the important features of a sampling system. 
The ideal or Dirac comb type of sampler produces a periodic output spectrum 
which has a fundamental shape corresponding to the spectrum of the input 
Signal. This type of output is illustrated in Figure 64-1. The effect of 
a finite linear sampling time in a mltiplying sampler is to introduce a 
seale factor of P(nw,) into the periodic spectrum of X(w- nw.) . This 
factor is not a function of frequency but its effect is to scale the ampli- 
tude of each term in the series. The shape of the pulse in this type of 
sampler does not affect the spectral shape of the baseband spectrum or any 
of the higher order spectra in the series. It does, however, affect the 
amplitude factor for each term. The sample-and-hold type of sampier, on 
the other hand, actually changes the spectral shape of the sampled input 
spectrum. This effect is easily accounted for by determining the Fourier 
transform of p(t). The output spectrum of a sampler is not a physical ob- 
servable and is usually processed by lowpass filtering to reproduce the 


original input signal. This filtering process is called reconstruction. 
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D. Data Function Reconstruction 


The input data function can be reconstructed at the output of the 
sampler by lowpass filtering the output spectrum. This is equivalent to 
convolving the output time function, y(t), with the impulse response, h(t) 
of the lowpass filter. The reconstructed signal is represented by an esti- 
mate of the original input, x(t). The reconstructed output is given by the 
convolution integral: 

A +o 
x(t) = [ A(dy(t- 1dr (6D.1) 
=o 

In the frequency domain, this filtering is represented by the multipli- 
cation of the output spectrum with the amplitude frequency response of the 


filter (Equation 6C.8): 


1 +o 
Xu = Wj ¥(w) = HCG) FE PCaw,)X(w- nw,) (6D.2) 
Ss 


Sa 
The convolution integral can also be written in terms of the original 


input signal and the sampling function as: 
A coe 
x(t) = HN h(t)x(t - tT)m(t - t)dt (6D.3) 
=o 
The solution for the output estimate, x(t), is now obtained by solving 
these system equations. . 
The convolution integral of Equation 6D.3 can be solved for the ideal 
sampler by replacing m(t) with a Dirac comb function and using the sifting 


property of the Dirac delta function as was done in Equation 6C.7. For the 


ideal sampler, the estimated output in the time domain becomes: 
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A +o 

x(t) = ss x(n.) h(t - nT.) (6D.4) 
This equation has an interesting mathematical form that is worth discussing 
because it gives insight into the sampling and reconstruction process. 
This series for the estimated function, x(t), uses a vector basis set which 
is derived from the impulse response of the lowpass filter. The amplitude 
coefficients are simply the data samples at times, nT. - It is significant 
to realize that any bandlimited signal, x(t), cam be represented by an in- 
finite series with amplitude coefficients corresponding to the samples of 
x(t) at times nt. and basis vectors corresponding to the impulse response 
of an arbitrary lowpass filter. The only requirement on the reconstruction 
filter is that it completely Peaueee the higher order spectral components 
of y(t). This requirement is satisfied if the transfer function for h(t) 
is zero for frequencies above Os 

A commonly used example for a lowpass filter that is frequently seen 

in textbooks is that of the ideal rectangular lowpass filter. The impulse 
response of this type of filter is obtained by taking the Fourier trans- 
form of a rectangle function with height Ny and "double-sided" spectral 


bandwidth 2. This transform is the well-known sinc function and can be 


written as: 
N sin(ut) 
oO 
h(t) = an 2m, ; ae | (6D.5) 
This impulse response is used in the series of Equation 6D.4 to give the 


well known result 


+0 sin2nf, (t- nT.) 


A 
: z RR Bia 
CE) OM y cece "a? Ine (ear) oe” 


where fy is the "single-sided" spectral bandwidth, in Hz, of the lowpass 
filter. 

In most cases, the results obtained for the ideal sampler can be di- 
rectly applied to "exact-scanning" and "flat-topped" samplers. Usually 
only the amplitude factor is affected. For exact-scanning, the estimated 
output spectrum is obtained by lowpass filtering the spectrum given by 


Equation 6D.2: 
Ra) = P(O)HCudE (a) mers (6D.7) 
s 


The inverse Fourier transform is taken to give the output in the time do- 
main. The ite equation will be the convolution of the impulse re-_ 
sponse of the filter and the input signal. The final result is given by 
Equation 6D.4 and the addition of a scaling factor to account for the 


finite gain of the sampler: 
+0 
tt) = 2 5 xcor yace-ar_) (6D.8) 
Ts n=<-0 s s 


P(O) is the Fourier transform of the pulse shape of the sampling function 
evaluated at zero frequency. For the previous example of a rectangle 
function, this factor is, P(Q) = AAT.- 

The flat-topped sampler introduces an additional complicating factor 
because the output power spectrum after lowpass filtering still contains 
the weighting runction P(w). This result is shown by filtering the spec- 
trum given by Equation 6C.19 with a lowpass filter to give the estimated 
output spectrum as: 


Xu) = HCa)¥(u) = 2 HCo)PCw)x(w) eas (60.9) 
: Ss 
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The solution in the time domain involves two convolution integrals and can- 
not be solved in general. If the width of the sampling pulse is suffi- 
ciently smail so that P(uw) is essentially a constant over the frequency 
interval 0 to Wy » the output in the time domain will be the same as that 
given by Equation 6D.8. This approximation is valid for most practical 
samplers. 

It is of practical interest to compute the spectral output of the low- 
pass filter by taking the Fourier transform of the estimated output given 
by Equation 6D.8. Since the only time dependent term in the equation is 
the time shifted impulse response, the Fourier transform is simply the in- 
finite series composed of the Fourier transform of h(t) and the shifting 


theorem phase factor: 


+o i 
Xu = FO way £ xcaryeInes lw! su, (6D.10) 
s n=<-0 


The infinite series part of (6D.10) is exactly the form of a digital 
Fourier series with infinite sampling. This important result shows how 
the continuous spectrum given by Equation 6D.7 can be obtained from the 
lowpass filter spectrum and the discrete samples of x(t). Both of these 
quantities are known in a practical sampling system. 

If the lowpass fiiter has a rectangular passband, its amplitude spec- 
tral function, H(w), will be a constant over the spectrum of x(t) and the 
output spectrum, X(w), can be computed, within a scale factor, by a digital 


Fourier series with infinite sampling. 
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E. Sampling Theory - Finite Record Lengths 


In practical sampling situations, the length of the input data 
function is either limited in time or else the data can be observed for 
only a finite time. This situation can be adequately modeled by using a 
sampling function which exists only over a finite time interval. Fora 
total of 2N+1 observations of the input signal, the sampling function is 


defined as: 


+o +N 
mt) =f p(t) £ 6(t- t-nT_)dt (6E.1) 


or in terms of the periodic pulse train as: 
+N 
m(t) = Z p(t-ntT_) (6E.2) 
n=-N 5 
The sampling function is periodic only over the interval of the defined 
pulses. In effect, the sampling function has been multiplied by a rec- 
tangular window function of width (2N+1)T.. This forces the amplitude 
spectrum of m(t) to be infinite and continuous. This effect does not 
cause a problem with the analysis of the system but does force the ob- 
server to accept, at best, only an estimate of the "true" input spectrum. 
The amplitude spectrum of the sampling function can be obtained by 

taking the Fourier transform of the finite pulse train of Equation 6E.2. 
The transform is simply the transform of the pulse, p(t), with a phase 
factor determined by the shifting theorem. This result may be written as: 

M(u) = P(u) F eJBWs (6E.3) 


n=-N 


If m(t) is modeled by an infinite pulse train multiplied by a window 
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function, the resulting amplitude spectrum is obtained by convolving the 
amplitude spectrum of the window function with the amplitude spectrum of 

the infinite series. The infinite series can be written as a Fourier series 
so its Fourier transform has already been given by Equation 6C.11. The 
Fourier transform of a window function of unity amplitude and width 
(2N+1)T. is the same as that for a rectangular pulse. This transform is 
given in Equation 6C.16 where A=1 and AT, = (2N+1)T.. The convolution 
integral can be solved by using the sifting property of the Dirac delta 


function to give: 


iat ere 
2N+1 oy, (RE Zt Ce aw) 
MC) eats Cal oN t 
Bee (AT, (w- nu) 


(6E.4) 


The amplitude coefficients, C.> are determined from the amplitude spectrum 
of the pulse shape as given in Equation 6C.14. When this substitution is 
made into (6E.4), the desired alternate form for the spectrum of the ml- 


tiplying function is obtained as: 


- 2N+1 
+2 sin(—3—)T, (w- nw.) 
M(w) = QN+1) 2 P(nw.)| aye (6E.5) 
Baer (YT, (w- nw) 


Equations 6£.3 and 6E.5 represent two equivalent series representations 

for the amplitude spectrum of m(t). Both of these representations have 
their usefulness in certain applications and insight into the properties 
of the sampling function can be obtained from both. The sampler output 
spectrum will now be computed using the representation of (6E.3). The out- 
put spectrum of the sampler, before filtering, is a series of 2N+1 func- 


tions obtained by convolving the spectrum of x(t) with the spectrum of p(t) 
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and a multiplicative phase factor: 
+N +o i 
Yu) = = sf KC P(w- rye TOM PT say (6E.6) 
n=-N e =-o 
If the sampler is ideal, the Fourier transform of the sampling pulse 
will be a constant and the convolution integral for the output spectrum 


of (6E.6) will be reduced to: 


+N 2; tage : 
¥(a) = $= Ze junts f xcayeti "saa (68.7) 
n=-. -o 


The integral in (6E.7) is the inverse Fourier transform of the amplitude 
spectrum of x(t). If we identify the inverse domain variable as nT,» the 


output spectrum can be simplified to the finite series: 
+N 7 
X(a) = _Z (ate junts (6E.8) 


In the frequency range of lw <u Y¥(w) is an estimate of the amplitude 
spectrum, X(w). As the number of samples of the input function becomes 
large, the estimate converges to the "true'’ value. This expression for 
Y(w) is identical to that of a discrete Fourier transform for a bandlimited 
spectrum. 

The output spectrum computed using Equation 6E.8 involves an infinite 
series of convolution integrals which convolve the sinc function in M(w) 
with the "true" spectrum X(w). This convolution in the frequency domain 
tells us that the spectrum of X(w) is smoothed by the finite observation 
process. Spectral smoothing reduces the spectral resolution of the obser- 


vation and limits the ability of the measurement process to resolve two 


56 


closely spaced line spectra. The problem of spectral smoothing will be 
discussed in detail in several later chapters. 

Other discussions of sampling theory as applied to commmication 
systems can be found in (Carlson, 1968, Ch. 7), (Thomas, 1969, Ch. 7), 


and (Schwartz, 1970, Ch. 3). 
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VII. THE FIRST PRINCIPLE OF DATA REDUCTION 


The reduction of real physical data often requires estimation schemes 
which include, either overtly or covertly, algorithms which modify the data. 

Often the selection of a particular estimation scheme results in observer 
bias being introduced into the data processing. A common example of this 
type of observer bias occurs in the selection of a lag window as is done 
in the Blackman-Tukey method of spectral analysis. Merely by selecting 
different types of lag windows, the observer can compute different esti- 
mated power spectra from the same observed data. In theory, we say that 
the autoeovariance function and the power spectral density function form 
a Fourier transform pair. When using a lag window, the inverse transform 
of the computed power spectrum will not "give back" the initial autoco- 
variance function. This type of processing is in violation of the First 
Principle of Data Reduction (Ables, 1974): 

"The result of any transformation imposed on the experi- 

mental data shall incorporate and be consistent with all 

relevant data and be maximally non-committal with regard 

to unavailable data." 

By the very nature of data processing and estimation it is ordained 
that the FIRST PRINCIPLE will be violated to some degree. The fact that 
all sampled data is finite in length causes this violation. Whenever 
real physical data is modeled by a mathematical function and a criterion 
of goodness is chosen, it follows that certain assumptions mist be made 
about how the model fits outside the domain of measurement. 


The first example of this is the modeling of a finite time series 


with a Fourier series. The complex amplitudes are computed by assuming 
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that the ieaete oF the observed series is the Fourier period, that the 
observed time series is periodic outside the interval of observation, and 
that the sampling rate is more than the Nyquist rate. If the observer 

has not sampled the data in such a way that these assumptions are valid, 
the computed complex amplitudes may be poor estimates of the "true" complex 
amplitudes. Observer bias has entered into the scheme by the method of 
data taking and by the tacit assumptions involved in using the digital 
Fourier transform. The irony of this situation is that the computed com- 
plex amplitudes can be used to generate the observed time series exactly. 
Of course we must realize that an infinite number of Fourier series may be 
used to describe a finite time series but only one was computed. 

A second example of time series modeling is the autoregressive-moving 
average time series. In maximum entropy spectral analysis, a least-square 
error criterion is applied to fit the observed time series. The assumptions 
in this method are that the process can be represented by an g getey 
autoregressive-moving average time series and that the observation time 
was long enough to permit an exact calculation of the autoregressive co- 
efficients. Also, it is assumed that the infinite time series can be 
computed from the estimated series. This is analogous to the assumption 
of periodicity in Fourier analysis. 

Although the first principle can never be ideally achieved it is worth 
striving for and any proposed data processing scheme should try to minimize 
its effects. Techniques which try to minimize the effects of data windows 


are referred to as "data adaptive spectral analysis" (Lacoss, 1971). 
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VIII. DATA ADAPTIVE SPECTRAL ANALYSIS 


The study of spectral analysis using only hypothetical or ideal data 
functions provides little insight into the practical problems involved in 
the study and analysis of empirical data sets. The reason for this is 
that a strictly mathematical analysis of data functions and spectral func- 
tions depends on the existence of infinite domains and infinite amounts of 
data. Attempts to modify the theory of spectral functions to apply to 
finite empirical data sets have been plagued by many problems. The theory 
that applies to infinite domains must be greatly modified to handle finite 
data sets. Spectral analysis methods for empirical data sets involve the 
definition and selection of various estimation procedures that, hopefully, 
result in realistic estimates of the "true" spectrum. A study of the 
various estimation procedures for spectral analysis constitutes the major- 
ity of effort in this type of research. A brief discussion of commonly 
used spectral analysis estimates will now be given and are used as an il- 
lustration of why data adaptive procedures are being sought. 

In most applications of spectral analysis, the goals are to determine 
the existence of "peaks" in the frequency spectrum, the location of these 
peaks, and their spectral power. The reason that the peaks are sought is 
that they contain the interesting information needed for scientific anal- 
ysis. Using definitions from information theory, a spectrum which is flat 
would be called a maximm entropy spectrum while a spectrum containing 


only one amplitude value would contain zero information. 
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Often, the amplitude spectrum is estimated directly by taking the 
fast Fourier transform of the empirical data. This procedure produces a 
discrete amplitude spectrum which is based on the assumptions that the 
data is periodic and spectrally limited both in the number of spectral 
components and the frequency of the highest component (See Chapter XII). 
In statistical studies of spectral functions derived from sample functions 
of a real process, the amplitude spectrum is avoided for theoretical 
reasons because it dias not exist or it will not give a power spectrum es- 
timate that converges. For this reason an alternate definition is often 
chosen. 

This alternate definition is the famous Wiener-Khinechine relations 
(Appendix I, Part F) using the autocorrelation function. This definition 
is preferred because of the previously given theoretical reasons. There 
are severe problems associated with the use of an autocorrelation function 
for empirical data. First, the estimated autocorrelation is badly trun- 
cated. If the spectrum is estimated by taking the Fourier transform of 
this truncated function, many additional frequency components are intro- 
duced that are not in the original spectrum. Second, this estimate is 
obtained from only one sample function of the process. For statistical 
reasons, many samples are needed to make the autocorrelation function con- 
verge to the "true" value. Third, the algorithm used to determine the es- 
timated function uses less data as the lag value increases. This causes 
the function to be a poor estimate for large values of lag. Practical 
schemes for estimating the power spectrum of a random process must take 


these undesirable effects into account and minimize their effect on the 
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accuracy of the estimate. 

The most commonly used method of spectral analysis of empirical data 
is the Blackman-Tukey method (Blackman and Tukey, 1958). This method uses 
the autocorrelation definition for the power spectral density of a random 
process. Their approach to the problems associated with the estimated 
autocorrelation function was to introduce a lag window function which 
would be multiplied with the estimated autocorrelation function. The lag 
window function was chosen so that it would weight the autocorrelation 
values in such a way that values near zero would be emphasized and large 
lag values would be removed or smoothed to zero. Their fundamental assump- 
tion was that the modified estimate of the autocorrelation function would 
give a respectable estimate of the smoothed values of power spectral den- 
sity even though the weighting process actually produced a poor estimate 
of the "true™ autocorrelation function. This assumption has been surpris- 
ingly effective in a number of practical applications and accounts for the 
success of this method. 

Although the Blackman-Tukey method is probably the best method in 
common use, it has serious drawbacks in many scientific applications. 
First, for limited amounts of data, the estimated autocovariance function 
is severely truncated and the process of "windowing" further reduces the 
range of useful lag values. The method recommends that only about 5 to 10 
percent of the estimated autocovariance function be used. This procedure 
effectively removes much valuable data that might otherwise be used in the 
estimation of the spectrum. Second, the selection of a lag window intro- 
duces bias into the data and tempts the observer to "select" a window func- 


tion that produces the "most agreeable" spectrum. The use of a lag window 
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function is also found to be objectional on theoretical grounds because 

it causes "negative" power spectra to be produced. Third, in many practi- 
cal situations, the goal of spectral analysis is to detect and resolve two 
closely spaced spectral peaks but the Blackman-Tukey method of using a lag 
window produces a smoothed estimate of the "true" spectrum resulting in a 
loss of spectral resolution. The most obvious questions to be asked are: 

1) How can the data be processed to use all of it and still avoid the 
problem of truncation and 2) How can the smoothing effect associated with 
"windowing" be minimized to produce the best spectral resolution? Data 
adaptive spectral analysis methods are an attempt to answer these questions. 

The discussion of the drawbacks in the Blackman-Tukey method should 
not be viewed by the reader as an attempt to discourage the use of this 
method. On the contrary, there are many practical situations where this 
method gives very good results. It is important that the reader be aware 
of the problems attendant to the use of a lag window and the possible al- 
ternatives to its use. As is often the case, the user will probably try 
several methods and compare the results with his a priori knowledge about 
the measured process. 

There has been mich work done on the selection of an "optimum" window 
function and many window functions are in common use (Otnes and Enochson, 
1972). The primary goals of this work have been to retain as much spectral 
resolution as possible and still minimize the production of "negative" 
power spectra. One cannot escape the fact that the concept of a window 
function will always be valuable in the discussion and applications of 


power spectrum estimation techniques. Windowing can, however, be ap- 
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proached in entirely different ways if alternate modeling techniques are 
employed such as those used in data adaptive spectral analysis. 

In recent years, data adaptive methods of spectral analysis have re- 
ceived increasing attention from experimentalists. These methods attack 
the problem of limited data and are producing surprisingly good spectral 
estimates using data that cannot be easily processed by the Blackman-Tukey 
method. Data adaptive methods are based upon assumptions which allow 
limited degrees of smoothing and extrapolation to be applied to the esti- 
mated autocorrelation function. Within the limits of the various assump- 
tions, this technique essentially produces an infinite function which can 
be easily handled by conventional theory. This approach essentially cir- 
cumvents the need to select a window function but has not eliminated the 
theoretical requirement that a truncated autocorrelation function needs 
some type of "windowing". The way in which a truncated autocorrelation 
function may be smoothed and extrapolated is governed by the criterion that 
is selected for the particular estimation routine used. Data adaptive 
methods are so named because the empirical data actually governs the opti- 
mization criterion used for smoothing and extrapolation. It can be said 
that the problem of windowing adapts itself to fit the nature of the data. 

Lacoss (1971) published a widely referenced paper in which he discusses 
the mathematical formulation of such data adaptive techniques as maximm 
likelihood and maximum entropy. His use of a matrix formulation of the 
methods adds a valuable dimension to their understanding and provides the 
reader with more theoretical insight than can be obtained by studying the 


algorithms only. This paper is highly recommended reading because it com- 
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pares the data adaptive methods with more "conventional" methods. 

This discussion has been given to provide the reader with the necessary 
insight into the problems associated with the estimation of power spectra. 
It should be emphasized again that the problem of truncated data is not 
considered by conventional theory and that various estimation techniques 
must be used. These estimation techniques involve either the direct use 
of a windowing function as in the Blackman-Tukey method or smoothing and 
extrapolation as done in the data adaptive techniques. The selection of a 
particular technique of spectral analysis is discussed in Chapter IX. The 


implementation of some of these various techniques is discussed in later 


chapters. 
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IX. METHODS OF SPECTRAL ANALYSIS 


The selection and application of a particular method of spectral 
analysis is a very complicated task and offers many challenges for the 
potential user. The development of a method usually begins by selecting 
known estimators and then modifying them to fit the conditions of the prob- 
lem. This process of development often requires a unique mixture of theory, 
art, and "luck" before a suitable solution is obtained. 

Estimators obtained from the theory of spectral functions can be 
greatly improved if as much information as possible is incorporated into 
the development of a technique. Physical boundary conditions, a priori 
assumptions, and other information should be used. Modifying the estimator 
in these ways can be thought of as a type of model fitting. It should be 
apparent that effort expended in obtaining the best starting model will pay 
off in producing the best final results. 

It is ironic that the selection of a "best" spectral estimator often 
depends upon a priori assumptions that may or may not be valid for the 
empirical data to be analyzed. For example, the computation of an ampli- 
tude spectrum by Fourier analysis techniques depends upon the assumption 
that the function is periodic and that it has been observed for exactly 
one Fourier period. This assumption is seldom true but the amplitude 
spectrum is estimated this way by most users. Another example is the 
representation of empirical data by an autoregressive series. It must be 
assumed that the process produces a sample function that is an autoregres- 
sive series before the autoregressive spectral estimator can be used. 


Again this assumption is seldom true but the autoregressive spectral esti- 
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mator is used with good success and can give very good approximations to 

the spectrum of a periodic function provided the autoregressive series has 
enough coefficients. The validity of these various assumptions will affect 
the accuracy of the spectral estimate and offers a basis for judging the 
"goodness" of a particular technique. The effect of the various assumptions 
will be discussed in detail in later chapters. 

A data function is characterized in the frequency domain by its ampli- 
tude spectrum or its power spectrum. These spectrums are estimated by ap- 
plying various filtering techniques and using che theory of spectral func- 
tions given in Chapter IV. Since spectral estimation theory is based on 
the use of filtering, Fourier transforms, and autocorrelation functions, 
the development of suitable estimators is primarily concerned with obtaining 
good approximations to the integrals in these techniques. Estimators for 
the amplitude spectrum are based on the discrete Fourier series or trans- 
form, Estimators for the power spectral density function are based on 
filtering and/or the autocorrelation function and its transform. The auto- 
regressive spectral estimator is based on the discrete form of the auto- 
correlation companion function discussed in Chapter XV. The various es- 
timators can also be described in terms of linear predictive filtering and 


recursive digital filtering. 
A. Computing Methods 


It is difficult to place the various spectral analysis methods into 
distinct categories because of the large number of variations that can be 


applied. For the benefit of the reader, a brief listing will be given in 
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an order that seems the most logical to the author. Other methods of 
presentation based on different viewpoints would be equally logical. The 
amplitude spectrum can be obtained by computing the Fourier transform of 
the data function. This scheme seems suitable and no other methods are 
commonly used. The computation of the power spectrum is quite a different 
matter. Most schemes are based on the autocorrelaticn function, recursive 
digital filtering, or the autoregressive spectral estimator. Methods which 
avoid the explicit use of a window function are called data adaptive methods 


(See Chapter VIII). Estimators for the power spectrum include: 


1. The Blackman-Tukey method (autocorrelation function, window func- 
tion). 
2. The direct Fourier transform method (amplitude spectrum squared). 
3. Recursive digital filtering. 
4. Data adaptive methods, 
a) maximum likelihood method. 
b) maximum entropy method, Burg algorithm. 
c) maximum entropy method, Levinson algorithn. 
5. Bartlett estimator. 
6. Autoregressive spectral estimator. 
These various estimators do not represent completely independent methods. 
For example, the data adaptive methods and the autoregressive method can 
be formulated with the same mathematics that is used to describe recursive 
digital filtering. Likewise, the autocorrelation function and lag window 


function can be represented by analogous equations representing the digital 
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filter. The Blackman-Tukey method could then be formulated with the proper 
selection of a recursive filter. The selection of a particular representa- 
tion often depends on the viewpoint adopted by the user or his estimation 


of the most "economical" approach. 
B. Data Collection and Experimental Design 


The observation and collection of data can have an important inter- 
active effect with the chosen method of spectral estimation, The experi- 
menter should consider the effects of data collection on the errors intrin- 
sic to a particular spectral estimator. An observation system model for 
spectral analysis studies is shown in Figure 9B-1. A hypothetical sample 
function, s(t), is used to model a ergodic real process. It is assumed 
that s(t) will generally contain both random and periodic components and 


is properly well-behaved so that its autocorrelation function and power 


spectrum exist and are well-defined. In theory, the "true" sample function 
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Figure 9B-1. Observation System Model for Spectral 
Analysis Studies 
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of a process is inaccessible to direct observation. This is because the 
observation process limits the. observation bandwidth and total number of 
data samples. This has the effect of limiting the spectral bandwidth and 
resolution of any spectral estimator. 

The inherent bandwidth limiting effects are modeled by a filter with 
impulse response, h(t). This filter may represent the time constants 
associated with probes, cables and other electrical conductors or it may 
represent the use of prefiltering for the sampler. Prefiltering is neces- 
sary for the successful operation of a practical data sampler. This filter 
is designed to have a sharp cutoff characteristic to limit the maximm 
spectral component that is transferred to the sampler. The filter should 
be designed so that it effectively attenuates all frequencies above the 
folding or aliasing frequency, f. = 7 The filter passband and the 
sampling rate can be properly Fre ee make the problem of aliasing 
negligible compared to spectral mixing errors in the estimator. 

The effects of finite data sampling are modeled by a miltiplying or 
"natural" sampler. The effects of data sampling on the spectrum of an in- 
put function were discussed in Chapter VI. The observable function, d(t), 
at the output of the sampler will be called the data function. This will 
be the function that represents the actual data used by the spectral esti- 
mation routine. The data function represents all that is observed about 
the generating process and its sample function, s(t). All estimates of 
the spectrum of the process will depend on the observed data in the data 
function. 

The spectrum of d(t) can be considerably different than the spectrum 


of s(t) if proper experimental design is not used. First, the observation 
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filter will pass only those spectral components of s(t) which are in the 
filter passband and will attenuate or reject those components outside the 
passband. The filter output is the convolution of the sample function, 
s(t), and the filter impulse response, h(t). If the spectral estimator is 
capable of producing highly accurate estimates of the spectrum of d(t), it 
may be necessary for the experimenter to deconvolve or "inverse-filter" the 
output to obtain a better estimate of the spectrum of s(t). Second, the 
sampler will introduce frequency mltiplication and some aliasing of the 
spectrum passed by the filter. Third, the limited number of available data 
samples will limit the spectral resolution of the estimator. All of these 
effects must be accounted for if acceptable estimates are to be obtained. 
Many of the problems caused by the observation and sampling of a 


process can be minimized by proper experimental design. For example: 


The filter bandwidth can be made wide enough to pass all signif- 


roy 


icant spectral components of s(t). 
2. The sampling rate can be made high enough to avoid aliasing 
problems. 
3. The output of the sampler can be digitally filtered so only the 


baseband spectrum (also called the principle alias) is determined. 


Even with careful experimental design, the estimated spectrum of a 
process can never converge to the "true" spectrum because of the intrinsic 
spectral estimation errors. A finite, sampled-data function will always 


be corrupted by aliasing, leakage, spectral smoothing, and statistical 


variability. 
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C. Data Preprocessing and Intrinsic Spectral Estimation Errors 


Empirical data that must be used for spectral studies often is cor- 
rupted with errors that are due to the observation technique or are in- 
herent to the estimation process. The user can minimize these effects by 
properly preprocessing the data and choosing an estimation method that will 
minimize intrinsic effects. The following data preprocessing techniques 
can be used, when needed, to improve the accuracy of a chosen spectral 


estimator: 


1. Wild point editing. 

2. Trend removal. 

3. Remove average value ("dc" value). 
4. Smoothing and decimation. 

5. Prewhitening. 


6. Rejection filtering. 


Wild point editing is necessary when the observation procedure is 
likely to introduce data values that are erroneous and not related to the 
"true" data. These types of errors are usually due to equipment malfunc- 
tions and gross errors committed by the observer. A "wild" data value can 
cause considerable trouble in an estimation routine if the amount of data 
is limited. For instance, a least-squares fit can be greatly influenced by 
a data value that is far removed from the mean. The least-squares method 
would place great importance on the value because it has a small probability 
of occurrence. The selection of an editing scheme is largely a matter for 


user definition and depends on the type of data to be collected. A scheme 
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that rejects data points with large variance is discussed in Otnes and 
Enochson (1972, p. 56). 

Trend removal is an euphemism for treatment of nonstationary data 
that has a mean value that varies over the interval of observation. This 
violation of the assumption of stationarity is not serious if the rate of 
variation of the mean is very slow compared with the rate of fluctuafion 
of the data. This slow variation in the mean value can be removed by 
model fitting with a low-order polynomial, partial sinewave, exponential 
function or other estimation schemes listed in Chapter III. Trend removal 
causes an irrevocable change in the low frequency spectrum of the data and 
cannot be used if low-frequency spectral estimation is important. Trend 
removal can be treated as a process of highpass filtering and appropriate 
filtering techniques can be applied. 

The removal of the average value from the data function is really a 
simple form of trend removal where the "dc" spectral component is "filtered" 
out. The accuracy of most spectral estimators can be improved by removing 
the average value because this minimizes the "spectral mixing" effect 
(Chapter XI). This "spilling over" or "leakage" of one spectral component 
onto the frequency of another is an important source of estimation error 
and "de" removal is almost always recommended. 

Smoothing and decimation can be applied to data functions that are 
sampled at several times the Nyquist rate. As discussed in Chapter VI, 
oversampling increases the spectral bandwidth and the total number of data 
samples. If the total spectral bandwidth is not needed to display the 


desired spectrum, it is often desirable to filter or smooth the unwanted 
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Spectrum. This will dei niee the et técts of spectral mixing and reduce any 
high frequency noise that may have been introduced in the measurement pro- 
cess. Decimation (Blackman and Tukey, 1958, p. 45) refers to the process of 
reducing the total number of data points by selecting only every second 

or third sample. This process has the effect of reducing the sampling 

rate and increasing the sampling interval to 2AT, 3aT, etc. Decimation 

is practical only when the data are highly oversampled and the desired 
spectral bandwidth is well-defined. Oversampling of data has some desir- 
able features such as improved estimation accuracy and minimization of 
aliasing problems that should not be overlooked. Decimation should only 

be applied when the reduction of the total number of data samples is ab- 
solutely necessary due to processing efficiency problems. 

One of the most frequent causes of errors in spectral analysis is 
that due to spectral mixing (called intermodulation distortion by Blackman 
and Tukey). This problem is especially severe when the data spectrum 
contains one or more peaks which have a high spectral power. Some of this 
power will "leak" into another part of the spectrum and give an erroneous 
contribution to the spectral amplitude. The leakage power can be so high 
as to cause the true spectrum to be completely "masked" by the leakage 
contribution. The spectral mixing problem is greatly reduced for a data 
spectrum which is essentially flat or "white". It is often observed that 
spectral estimators give the most accurate results for a "white noise" 
spectrum. Prewhitening refers to the process of adding a synthetic signal 
to the measured data signal that will add to the data spectrum to make it 


essentially flat. This "“prewhitened" signal is then spectrum analyzed 
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and the resulting spectrum corrected for the added spectrum. 

Prewhitening essentially corrupts the data with additional noise and 
may cause the "purist" to reject its use. This should not be the case, 
however, because it is designed to improve the estimation and not the 
theory of spectral daanesie. There is a strange aspect of prewhitening 
that needs further discussion. This involves the design of a filter or 
other processor that is used to prewhiten the data. One might suppose 
that the technique used to determine the required synthetic spectrum might 
also be suitable for estimating the data spectrum directly. This is in- 
deed the situation and the data adaptive techniques that employ recursive 
digital filtering are used to compute a "prewhitening filter" that is used 
directly to estimate the data spectrum. The prewhitening filter concept 
is the basis for the autoregressive spectral estimator and the maximum 
entropy spectral estimator. 

Rejection filtering is yet another attempt at reducing the effects of 
spectral mixing. This preprocessing technique should be used if the data 
spectrum has an unusually strong peak. The amplitude of this peak can be 
reduced by notch filtering to make it comparable with the rest of the 
spectrum. The application of this technique usually requires at least 
one "run" of the data function through a spectral estimator that will 
detect this strong peak. The characteristics of the rejection filter 
must be retained in the processor so that the spectral peak may be "'recon- 
stituted" in the final estimate. 

The preprocessing of empirical data and the careful selection of a 
spectral estimator are necessary for successful spectral analysis. These 


processing and selection techniques are designed to minimize the effects 
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of intrinsic spectral estimation errors. Intrinsic errors are those 


which are present for all estimation methods. These errors often do not 
appear explicitly in the analysis of the estimation method and may cause 
unexpected problems if the user is not aware of them. The most common 


errors are listed as follows: 


1. Aliasing (undersampling). 

2. Spectral mixing (intermodulation distortion or "leakage"). 
3. Spectral smoothing. 

4. Statistical variability. 


5. Lack of stationarity. 


Aliasing or frequency fold-over is caused by undersampling or sampling 
at a rate that is less than the Nyquist rate. Aliasing will be present to 
some extent in all practical sampling systems. The undesirable effects of 
aliasing can be minimized by the use of sharp cutoff filters and adequate 
sampling rates. Every attempt should be made to make the effects of 
aliasing negligible compared to the effects of spectral mixing. Aliasing 
is discussed in detail in Chapter VI. 

Spectral mixing is the most severe problem in spectral analysis. 

Most estimators are designed to minimize its effects but it still causes 
problems when the data spectrum has one or more high spectral peaks. This 
problem can be described by a "leakage" of the power at one spectral 
frequency into another frequency. Lag window functions that are used in 
the Blackman-Tukey method are usually chosen to minimize leakage. The 


term leakage is derived from the analogy between a window function and the 
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field pattern of an antenna. The one-dimensional aperture distribution 
and the far-field power pattern for a beam antenna are reciprocal Fourier 
transforms as are the lag window function and the spectral window function. 
The minor lobes of the eee cause "leakage" of the power that is wanted 
in the main beam. Likewise, the lag window function causes leakage of 

the spectral power at the observed frequency into adjacent frequencies. 
The problem of spectral mixing is present whenever the observed data func~ 
tion is truncated. This is true for all known estimators. In most prac- 
tical schemes, the problem of leakage can be traded off against the prob- 
lem of spectral smoothing. 

Spectral smoothing is also caused by the truncation of the observed 
data function. The lack of enough data to resolve closely spaced spectral 
components causes two nearby peaks to be "smeared" together. The only 
solution to the smoothing problem is to observe the data function for a 
longer period. In the Blackman-Tukey method, spectral smoothing is an 
obvious result of the convolution of the spectral window function with the 
"true" spectrum. As the period of observation increases, the spectral 
window function becomes very narrow and approaches a Dirac delta function. 
When this happens, the estimated spectrum converges to the true spectrum. 
In filtering and date adaptive techniques, the spectral smoothing effect 
is not so obvious. It is introduced indirectly when the data function is 
smoothed and processed to obtain estimates of the filter coefficients or 
autoregressive coefficients. Data adaptive spectral eetinaeees such as 
the maximm entropy estimator are superior to the Blackman-Tukey method in 


their ability to minimize the effects of spectral smoothing and to give 
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better spectral resolution. This improved resolution is due to the fact 
that the adaptive methods utilize all of the data while the Blackman-Tukey 
method is restricted to about 5 to 10 percent of the total lag values. 

The reason for limiting the useful range of lag values in the Blackman- 
Tukey method is to minimize the effects of statistical variability in the 
estimate. 

The statistical variability of the estimate is caused by the finite 
observation time for the data function. The time autocorrelation function 
definition for power spectra is valid only if the process is both station- 
ary and ergodic. The definition of the time autocorrelation function 
depends on an infinite time. If the data function is observed for a 
finite time, the time autocorrelation function is only an estimate of the 
"true" function. This introduces statistical variability into the esti- 
mated autocorrelation function that directly carries over into the esti- 
mate of the power spectrum. The effects of statistical variability can 
be reduced only by making many observations of the process to obtain. and 
"average" autocorrelation function. A large number of truncated estimates 
of the autocorrelation function will sufficiently reduce the effects of 
Statistical variability but will have no effect on the spectral smoothing 
problem. Spectral smoothing can only be reduced by increasing the length 
of time the samples are observed. 

If an observed process is not stationary, most spectral theory and 
commonly used spectral estimates will not apply. The power spectrum of 
the process will be a function of time and can only be obtained from the 


ensemble definition of the autocorrelation function. If the time variation 
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is slow enough so that the spectrum is essentially constant for the time 
interval of observation, a “quasi-stationary" estimate can be obtained. 
In this instance, the deleterious effects of a short observation time 
must be traded against those of a time varying spectrum. The display of 
time varying spectra can be accomplished by a power spectral density pro- 
file plot (Otnes and Enochson, 1972, p. 415). 

The multitude of effects involved in the preprocessing of the data 
function, the sampling process, and the intrinsic spectral estimation 


errors must be collectively considered in the process of method selection. 
D. Method Selection 


Method selection should begin with a detailed study of the physical 
. process that generates the data function. The purpose of this initial 
study is to discover and incorporate as many constraints as possible into 
the process mcdel. The constraints will help to improve the accuracy 

and /oe efficiency of the estimator and often mean the difference between 
success and failure. After the constraints are determined, the next step 
is to see how the various possible estimators compare in efficiency, 
accuracy, bias, and other appropriate characteristics. For these con- 
parisons, the user must define a criterion of goodness and determine what 
estimator is "best" for his application. This determination will often 
involve a detailed mathematical analysis of the estimator coupled with 
several tests using hypothetical data semples from a "known" process. 
Finally, once an estimator is chosen, the spectra produced by this esti- 


mator must be interpreted as to its accuracy and ability to minimize such 
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effects as aliasing, leakage, and spectral smoothing. 

It is very difficult to develop a step-by-step procedure for the 
spectral analysis of data. Each problem presents its own unique set of 
boundary conditions and each experimenter must choose a criterion of 
goodness that is suitable for his purpose. Spectral analysis "by the 
numbers" is not possible under these circumstances. Some general guide- 
lines are possible if the reader is willing to accept loosely defined 
boundary conditions and criteria of goodness. These guidelines will now 
be presented along with an examination of the more fundamental processes 
involved in spectral estimation. These fundamental characteristics are 
discussed in Section E of this chapter. 

A flowchart of some commonly used spectral analysis schemes is 
shown in Figure 9D-l. The top flowchart shows the possible theoretical 
analysis methods for the theoretical data function, s(t). There are 
essentially three ways of obtaining a spectral function. First, for a 
periodic function, the amplitude spectrum is determined by the complex 
amplitude coefficients of the Fourier series. The spectrum is bandlimited 
and discrete for all real data functions. The amplitude spectrum can be 
used to reconstruct the original data function. Second, the spectrum of 
a single time pulse can be specified by taking the Fourier transform. The 
resulting amplitude spectrum is both continuous and infinite. As before, 
the time pulse can be reconstructed by taking the inverse Fourier trans< 
form. Third, for random processes, the power spectrum is defined as the 
Fourier transform of the autocorrelation function of the process. For 


many theoretical data functions that represent sample functions of a real 


81 


SpoyjeW syst, Buy [eajoeds Jo AABYyoMoTA WY °1[-a6 ean3zy 


1OVBUTISY [eazoeds 


SRUDIPFOTIJION VATSssaaZe10gny pouryay setzasg 


DAT sserZorz0gny Bj BUTIASY BATSS9IAZIIOANY 
pozenbs WIOJSUBAT ABTANOg pouren WAossuPray 
apnj use, sqanjposqy a79198Tqd AVayAnoY 39a; 


A0RBUTISY [vaqoedsg 
DAT ssIIZ9A0Qny 


uo zB [odeaqx| pue Sutyjoous 
UoFVOUNT UoTIeTLVAIODOINY 


BAUIFOFIIION 


sonbtuyoa,, 
DAT SsIAZVIAORNy 


sATadepy ejeq 


wnajoads 
19M0g 
pedeutjsy 


XOMOd ABITFA [BIOL 


é BuyAeI TTA 
Xo and3no s1zenbs-ueay 


AO UOTFANTOAUOD 


SufAVITTA 1[BIF3ta 
DATSANIIY 


wnajoeds 


1aMod uoT_OUNT, AOpuTM BoUP TABAODOINY poyren 
pojeutizsy 3e] pejewtyzeq AdyN]-usuyjovy gq 
uoFaound 


wnaijoads 19M0g WIOFSUBAT, A9TANOY 


snonuyquog 
wnajoedg epnattduy 


UOT IBTIALODORNY 


sostnd owty 1z07) 
umijoeds apna ttduy 


WIOFSUBAL TITAN] 


wunzjoedg o7J0198Tq poz pT] pug 
SRUDPFOTJJIOD A9pPanog 


(suoy o}Fpotzed 207) 


S9TAVS AVTAnog unijzoeds apnayt {duy 


a)P 
uofzouny e7eq 
leopazauy 


uof}oUNT BIeq 
[BOT QeA00Ny, 


(3)8 
UOTIOUNnY] BVaeg 
[eof AoA0aNUT, 


82 


process, the amplitude spectral function is not defined because the Fourier 
transform of the data function does not exist. For these functions, the 
original time function cannot be reconstructed from either the autocorre- 
lation function or the power spectrum. The autocorrelation function def- 
iereiod for the power spectrum is widely accepted as the most useful for 
random processes and can also be applied, if one accepts the use of the 
Dirac delta function, to periodic functions and single time pulses. 

The bottom flowchart in Figure 9D-1 shows several commonly used 
spectral estimators and their most important processing steps. The 
Blackman-Tukey method, the data adaptive techniques, and the autoregres- 
sive spectral estimator all use the autocorrelation function definition 
for the power spectrum. The recursive digital filter methods defines the 
power spectrum by computing the power output of the filter. The direct 
Fourier transform method defines the power spectrum by computing the 
absolute magnitude squared of the complex Fourier coefficients obtained 
by taking the digital Fourier transform of the empirical data function. 

A typical empirical data function is made up of 2N+1 samples of the 
theoretical data function. Since this time series is limited both in the 
total number of samples and the length of observation time, it can be 
exactly represented by a Fourier series with 2N+1 complex amplitude co- 
efficients (double sided spectrum). The coefficients can be used to de- 
termine both the amplitude spectrum and the power spectrum. The accuracy 
of this estimate will depend on the nature of the "true" data function. 

Likewise, this same data function can be represented by an autore- 


gressive series of any order from order one to order 2N+1. The power 
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spectrum can be estimated by using the autoregressive spectral estimator 
or the autoregressive series can be used to generate the discrete auto- 
covariance function and then the discrete Fourier transform used. This 
scheme can be applied to any truncated and discrete data function. The 
accuracy of the spectral estimate will again depend upon the nature of 
the "true" data function. 

In still another way, this empirical data function can be used to 
determine the estimated autocovariance function for use in the Blackman- 
Tukey method of spectral estimation. The accuracy of this method will 
depend upon the amount of data available for obtaining the autocovariance 
estimate and the window function selected for smoothing. As the amount 
of data becomes large and the observation time increases, the Blackman- 
Tukey estimate will converge to the "true" autocorrelation function def- 
inition of the power spectrum. This property of the Blackman-Tukey method 
makes it the "best" estimator in terms of adherence to the autocorrelation 
definition of power spectra. 

It may be difficult for the reader to accept the fact that these 
methods may all be considered “exact™ for the purposes of representing 
the empirical data function. In this sense, any of the methods could be 
used to define the spectrum of an empirical data function. The only way 
to validate a particular method as being the "best" estimator is to apply 


some a priori knowledge about the process that generated the data. In 


other words, if absolutely nothing is known about the process that gen- 
erated the empirical data function none of the spectral estimators can 


be selected as a "best" estimate. 
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The a priori knowledge needed to select a best estimator does not 
have to be very detailed or specific. Usually, a very general assumption 
can be used that will help. As an example, we consider the assumptions 
needed to justify the more common spectral estimators. To use the auto- 
regressive spectral estimator it is necessary only to assume that the 
data function can be represented by an autoregressive series because the 
process is autoregressive. This assumption is surprisingly good for many 
practical data functions. The digital Fourier series representation is 
exact if the data function is assumed to be periodic and if the period of 
observation is exactly equal to one Fourier period. The maximum entropy 
estimator is valid if it is assumed that the process is autoregressive 
and that the resulting spectrum has maximum spectral entropy. An addi- 
tional boundary condition on this estimator is that the autocorrelation 
function and power spectral density function be exact reciprocal trans- 
forms (no window function). 

The Blackman-Tukey estimator differs from most of the other estimators 
in that it converges to the "true" spectrum for random processes as in- 
creasing amounts of data are taken. This circumstance is brought about 
because of the choice of the definition for the apparent autocovariance 


function, The assumptions required for validating this method are: 


1. The Fourier transform of the modified apparent autocovariance 
function is a respectable estimate of the smoothed valued of the 


"true" spectrum. 


2. The statistical variability of the estimate is small enough to 


be acceptabie. 
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In theory, the Blackman-Tukey estimator is capable of giving the best 
spectral estimate when the data function is well-sampled. 

When nothing is known about the process that generated the data 
function, it is safest to select an estimator that requires the weakest 
assumptions about the process. Recent work by researchers in geophysics, 
astronomy, and other scientific areas that require the tools offered by 
digital spectral analysis has shown that the maximum entropy spectral 
analysis method offers higher spectral resolution and improved estimation 
accuracy over previous methods. If large amounts of data are available, 
the Blackman-Tukey method can also be successfully used. 

Figure 9D-2 is a data processing flow diagram that illustrates how 


data might be processed to obtain a spectral estimate. The diagram is 


based on the consideration of several characteristics of the spectral 


analysis problem. These are: 


1. The process model. 
2. Discrete or continuous output. 
3. Amount of available data. 


4. Sampling rate. 


5. Desired accuracy of estimate. 


"Real data" represents the actual signal output from the process. If it 

is continuous, an analog spectrum analyzer can give the desired spectrum. 
If the output is analog but has a finite duration, the data record will 
usually be sampled and preprocessed. The proper experimental design should 


be observed so that the finite record is sampled above the Nyquist rate 
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Figure 9D-2, Data Processing Flow Diagram for Illustrating a 
Possible Scheme of Spectral Analysis 
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and prefiltering should be used. If the data output are already in the 
form of a time series, the experimenter cannot exercise control over the 
sampling process. If the data are undersampled, the spectrum of the time 
series will be aliased and spectral accuracy may be seriously degraded. 

If the data is periodic or "almost" periodic, it can be processed 
using the amplitude spectral estimator. A knowledge of the period of the 
function allows the use of some strong model constraints that can improve 
the statistical variability of the estimate by reducing random effects 
due to sampling or noise. If the data is not periodic or if the noise 
level is so high as to make the data function appear to be random, the 
data must be treated accordingly. 

Figure 9D-3 shows one possible scheme for selecting a spectral esti- 
mator for the empirical data function obtained from a random process. 

The decision as to what kind of estimator to use essentially depends upon 
how much is known about the process model and how mich data is available 
for processing. A selection of possible estimators is shown. If a large 
amount of data is available for analysis and the process model is unknown, 
the Blackman-Tukey method can be used to give the most accurate estimate. 
The computational efficiency will be considerably reduced from the direct 
FFT method but this sacrifice may be desirable if the experimenter needs 
the improved estimation accuracy. 

If large amounts of data are available, there is a scheme that appeals 
to the author that is not shown in the diagram. The scheme essentially 
would be used when high accuracy estimates are needed. The Blackman-Tukey 
estimator would be used to make iterative calculations of the power spec- 


trum as the amount of data is increased for each estimate. The power 
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Figure 9D-3. Data Processing Flow Diagram for Illustrating a 
Possible Scheme of Spectral Analysis for a 
Random Process 
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spectrum would be seen to converge and the rate of convergence and the 
variance of the final estimate would give a measure of the statistical 
variablilty of the estimate. This variability could be traded off against 
the resolution of the selected spectral window to give a user defined 
"best" estimate. The scheme can be described as the use of smaller sub- 
sets of data to obtain an estimate of the variance of the spectral esti- 
mate that used all available data. The author has not seen this scheme 
developed in the literature and has not had time to develop it in this 
research. Further study could be done to develop criteria for evaluating 
the limits on statistical variance and spectral resolution. 

The ultimate decision about the use of a particular spectral estima- 
tion scheme must be made by the user after serious consideration of the 
need for the estimate and the scientific consequences or conclusions to be 
made from the estimate. The occasional or casual user of spectral analysis 
and spectral estimation methods does not usually have the time to devote 
to a detailed analysis of possible estimation schemes. As a consequence, 
the casual user should be warned against placing great scientific impor- 
tance on the results of the estimate. He should use well-developed esti- 
mators as discussed in this chapter or as given in Blackman and Tukey 
(1958) or Otnes and Enochson (1972). Any presentation of an estimated 
spectrum and scientific conclusions based on that estimate should be weli- 
supported by a discussion of the details of the estimation scheme used. It 
is especially important to list all assumptions necessary to make the esti- 


mator valid. 
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It would appear to this author that maximum entropy spectral analysis 
will become an increasingly important technique in future years. Its a- _ 
daptibility to the data and its improved spectral resolution over other 
methods should establish it as the method most recommended to the casual 
user. It is especially useful when the amount of experimental data is 
limited. At the present time, the practical implementation of this tech- 
nique is not well-discussed in the literature. The reader is referred to 
Chapter XV and Appendix III for a further discussion. 

The experienced or serious user of spectral analysis theory and theory 
of spectral estimators must devote a considerable amount of time to the 
study of estimation theory and the criteria of goodness associated with the 
selection of a "best" method. He must especially be familiar with the 
strengths and weaknesses of each method and the attendant assumptions. In 
most cases, more than one spectral estimator should be used to analyze the 
empirical data function of a real process. The use of more than one esti- 
mator can help to reduce the effect of intrinsic estimation errors and 
provide more confidence in the estimate. 

A typical scheme for analyzing the spectrum of an unknown process is 
shown in Figure 9D-4. This scheme employs more than one kind of estimator 
and illustrates what the author considers a reasonable approach to a de- 
tailed spectral analysis study. A theoretical data function is prefiltered, 
sampled, and preprocessed to obtain an empirical data function. This em- 
‘pirical data function is used to do a spectral study of the process. Since 
the properties of the process are unknown, a process model cannot be de- 


veloped and estimation techniques that do not depend heavily on a particu- 
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Figure 9D-4. A Typical Scheme for a Detailed Analysis 
of the Spectrum of an Unknown Process 
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lar model should be used. First, the spectrum is estimated by the direct 
FFT method. This estimator is very computationally efficient and can 
provide a good first idea of the spectral shape. If the data is sufficient 
to warrant its use, the Blackman-Tukey estimator should be tried as it will 
give an estimate that converges (In theory, the direct method cannot). If 
the spectrum contains spectral peaks that may not be resolved, the spectral 
resolution can be improved by using the maximum entropy method. The spec- 
trum can be computed by either the FFT or by using the autoregressive 
spectral estimator. To reduce computation time, the autoregressive spec- 
tral estimator need only be evaluated at frequencies near the spectral 
peaks. The results of these estimations should be compared and additional 
processing applied as needed. 

If the data is very limited, it appears that the use of the maximm 
entropy spectral estimator is the best choice for obtaining maximum spec- 
tral resolution. A summary of the fundamental properties of spectral 
estimators will help to illustrate their applicability and help the user 


to make a wise selection. 
E. Fundamental Characteristics of Spectral Estimators 


Spectral estimators are usually developed by starting with the various 
theoretical definitions for a spectral function and Soaiyian -suteahie es- 
timations and aimaeetent approximations. A spectral estimator is usually 
designed to minimize the intrinsic estimation errors caused by finite data 
sampling. The various methods appear to differ because they represent 


different viewpoints concerning the "best" way to represent empirical data. 
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These different viewpoints usually are the eeauie of selecting different 
criteria of goodness and starting with different a priori assumptions. By 
carefully analyzing the commonly used spectral estimators, one can reach 
the conclusion that they do not represent entirely different approaches to 
the estimation problem. Because they are all derived from the same theo- 
retical basis, the various estimators share many common characteristics. 
When developing a suitable spectral estimator, there are essentially 
three commonly held viewpoints concerning the best way to treat the problem 
of finite data. These viewpoints nave been developed by the author from 
reading the literature in this area and they may not represent the break- 
down that another author might choose. This author believes that this 
separation adapts itself nicely to the discussion of method selection. 


The three methods are: 


1. "Lag windowing" of the estimated autocorrelation function. 
2. Smoothing and extrapolation of the autocorrelation function. 


3. Recursive digital filtering. 


The use of a lag window function to give a smoothed estimate of the power 
spectral density function represents the approach taken by Blackman and 
Tukey (1958). The use of smoothing and extrapolation to obtain an estimate 
of the autocorrelation function is the method chosen by those who prefer 
data adaptive methods which do not have fixed window functions. Data a- 
daptive methods such as maximum entropy and maximum likelihood (Lacoss, 
1971) smooth and extrapolate using their respective criteria. The trun- 
cated autocorrelation function can also be smoothed and extrapolated using 


purely mathematical means such as model fitting with some exact function. 
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This can be done using the straightforward techniques common to numerical 
analysis (Chapter III) or by using more exotic orthonormal expansions such 
as angular prolate spheroidal wave functions (Veltman et al., 1972). Re- 
cursive digital filtering gives an estimate of the frequency spectrum by 
computing the average power over the bandwidth of the filter. The filter- 
ing method adds the flexibility of choosing the bandwidth of the estimate 
(within certain limits) but still suffers from problems such as leakage 
and computational efficiency. Recursive filtering has the tutorial ad- 
vantage of being able to represent a wide variety of discrete spectral 
estimators including the important maximum entropy estimator. 

The use of a lag window and the autocorrelation function definition 
of a power spectrum is treated in great detail in Blackman and Tukey (1958). 
A brief summary of this method showing how the use of a window function 
reduces the effects of statistical variability and produces a smoothed 
estimate of the spectrum will now be presented. In discussing the Black- 
man-Tukey method we will use the autocovariance function instead of the 
autocorrelation function. 


An unbiased estimate for the autocovariance function (Appendix I, 


Part I) is defined by 


tae |r|) /2 
C.(1) = Zl x(t = 1/2)x(t + 7/2)dt (9E.1) 
“he |r|) /2 


where x(t) is a sample record of the process that has been observed for a 


time th (7) is called the apparent autocovariance function because it 
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is obtained from only one sample realization of the process. For each 
observed record, x(t), there will be a corresponding function, ci). 
Because cj) is an unbiased estimate, its expectation value for lag times 
between +i will be equal to the "true" autocovariance function, C(r). If 
several sample records are available, the corresponding apparent autoco- 
variance functions can be averaged to give a better estimate. Lag times 
near zero will show the least statistical variation because the most data 
is used for the estimate. As the lag value approaches tty the amount of 
data used for the estimate becomes very small and the statistical variance 
becomes very large. 

To avoid the use of lag values with large variances and to reduce 
problems associated with the truncation of the autocovariance function, 
Blackman and Tukey chose to multiply the apparent autocovariance function 
by a lag window function, D(t). The resulting function is called the mod- 


ified apparent autocovariance function: 
E¢r) = D(C (a) (9E.2) 


Although it is purely arbitrary, the lag window function is usually chosen 
so that it will give the most weighting to lag values near zero and 
smoothly reduce the weighting factor to zero at lag values approaching 5 


to 10 percent of the total record. 


For this discussion, ar) will be called the estimated autocovariance 
function because it is this function that is used to obtain the estimate 
of the power spectrum. To examine the accuracy of this estimate, we first 


take the expectation value of &¢1) as given by: 
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E{EC1) } = D¢TIELC, (1) } = D¢T)CCn) In| <z, (95.3) 


Since, by definition, the expectation of c¥(1) is equal to the "true" 
autocovariance for lag values between +T,, the expectation of Le) is 
equal to the "true" autocovariance multiplied by the chosen lag window 
function. 

The estimated power spectral density function is obtained by taking 


the Fourier transform of the expectation value of C7). The result is: 
Fielloy = fF (em y} = a8} (9E.4) 


P(E) is the Fourier transform of one particular estimate of the autoco- 
variance and E{P(£) } is the expectation value of all possible realizations. 
The Fourier transform of the expectation value of eo) is equal to the 
Fourier transform of the product D(1)C(t) as given in Equation 9E.3. The 
transform of the product is the convolution of the transforms of D(T) and 
C(t). The final result is that the expected value for all possible real- 
izations of the estimated spectrum is equal to the convolution of the 


"true" power spectrum with the spectral window function Q(f) 
feo 
E{P(£)} = [ QC£-AP(A)AA (9E.5) 
at 


where Q(f) is the Fourier transform of the lag window function (Appendix I, 
Part I). The best we can do with a finite data function is to give a 
smoothed estimate of the "true" power spectrum. Increasing the number of 
data records will help te reduce the problem of statistical variance but 
only an increase in observation time can help to improve the spectral 


resolution. 
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Under certain circumstances, it is theoretically possible to solve 
the convolution integral (deconvolution) to give the "true" power spectrum, 
P(£), when the spectral window function, Q(f), is known. For most prac- 
tical analyses this is not done because, with only a few sample records, 
the error due to statistical variance is great enough to more than offset 
any improvement derived from deconvolution. 

A simple example will be given to illustrate the smoothing and leakage 
problems associated with the use of a lag window. For simplicity, we 
choose a periodic sample function, x(t), made up of three sinewave com- 
ponents. Many. sample observations of x(t) are taken and the corresponding 
apparent autocovariance functions computed. These values are then averaged 
to produce a close estimate of the "true" autocovariance function for lag 
values between +T,. This averaging reduces the statistical variance for 
the apparent autocovariance function. The "true" autocovariance is not 
known for lag values greater than Ty because the sample function was not 
observed for time intervals greater than this. 

The average (or expected) value for the time autocovariance function 
is shown in Figure 9E-1. This truncated function is multiplied by a 
rectangular window function to produce the estimated autocovariance func- 
tion, C(1). The width of the window function must be chosen narrow enough 
to minimize the effect of statistical variation in the observed sample 
functions. Blackman and Tukey recommend that the width of the lag window 
be no more than 5 to 10 percent of the total lag record. In our example, 


the width is much wider because the statistical variance has been reduced 


by averaging multiple observations. For this example and others to follow, 
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continuous functions will be illustrated. Usually the actual data 
processing will be done digitally but the continuous functions more clearly 
illustrate the results without the additional complication of sampling. 

The spectral functions associated with the Blackman-Tukey method and 
this example are shown in Figure 9E-2. The effects of windowing and 
truncation can be illustrated by comparing the “true” spectrum, P(f), with 
the estimated spectrum, B(£). The estimated power spectrum has been ob- 
tained by convolving the spectral window function, Q(f), with the "true" 
spectrum, P(f). The spectral window function is a sine function obtained 
by taking the Fourier transform of the rectangular lag window function. 

The estimated spectrum is the superposition of many sinc functions. Each 
line spectrum in P(f) will generate a sinc function of corresponding 
amplitude in the estimated spectrun, P(£). These sinc functions are shown 
separately in the last plot of Figure 9E-2. Spectral smoothing has caused 
the two closely spaced components at fy and 7) to be smeared together. The 
smaller spectral component at f, has been obscured by this loss of spectral 
resolution and most likely would go undetected. Also, the spectral com- 
ponent at f, has been smeared out and leakage has caused considerable error 
in the estimated amplitude of the lower frequency components. Not shown 

is a de component in the spectrum caused by the net average value in the 
estimated autocovariance. 

The window function has also introduced some negative power spectra. 
"Negative power" is not possible physically and its presence in this esti- 


mation routine is one of the major complaints of the users of this method. 


The generation of negative components in the power spectrum can be reduced 
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E{C (1) } =C(r) 


D(1r) 


Figure 9E-1. The Estimated Autocorrelation Function Obtained from a 
Rectangular Lag Window 
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F (c(t) } = Pc£) 


Q(f£) 


Figure 9E-2, Example Spectral Functions that Illustrate the 
Blackman-Tukey Method 
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by the proper selection of a lag window function. The rectangular window 
function is seldom used because it causes such a severe leakage problem. 
It does, however, have the best spectral resolution. Other window func- 

tions such as the Hanning window or Hamming window (Otnes, 1972, p. 258) 

have mich smaller leakage with only a small sacrifice in spectral resolu- 
tion. 

In many ways, the Blackman-Tukey method of spectral analysis can be 
viewed as a "fix-up" approach for handling a "difficult" apparent autoco- 
variance function. They use essentially a three step opredch to obtain a 
spectral estimator. First, they define one estimate of the time autocor- 
relation function that they call the apparent autocovariance function, 
Co(7). While this estimator has desirable theoretical properties, it also — 
has serious practical shortcomings. Second, the practical shortcomings of 
this estimator ‘are minimized by using a lag window to obtain a second es- 
timator for the "true” autocovariance function. This second estimator, 
which Blacknan and Tukey call the modified apparent autocovariance function 
and the author calls the estimated autocovariance, is the function that 
must represent the "true" autocovariance function for spectral estimation 
purposes. .Third, the Fourier transform of the estimated autocovariance 
function, én), is used to obtain the estimated spectrum. This three step 
approach has good and bad aspects that will now be summarized. 

For theoretical purposes, the Blackman-Tukey definition of the apparent 
autocovariance function has many desirable characteristics needed for a 
good estimator. It converges to the "true" fiadeton as the record length 


becomes infinite and hence is unbiased. Also, its expectation value is 
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equal to the "true" function for lag values between +T.. 

For practical purposes, these desirable characteristics are often 
overshadowed by the problem of statistical variation. This can introduce 
a "de" component not present in the "true" function. It can also make the 
estimate very poor for large values of lag. This happens because the es- 
timator (Equation 9E.1) uses very little data for lag values approaching 
#Te 

A lag window function can be used to reduce the effects of statistical 
variance by removing the larger lag values. This effectively reduced the 
statistical variance in the estimated spectrum but it also introduces 
spectral smoothing effects. The selection of a suitable lag window be- 
comes a tradeoff choice between spectral resolution and statistical vari- 
ation. A lag window removes valuable data that, although it may have a 
large variance, could still be used to give some improvement in the spec- 
tral estimate. | 

A technique that avoids the problem of choosing a lag window function 
and that attempts to use all of the observed data has a lot of practical 
appeal. Such techniques come under the broad category of data adaptive 
techniques and apply methods of smoothing and extrapolation to obtain an 
“acceptable" estimate for the autocovariance function. Conceptually, 
smoothing and extrapolation are more difficult to justify than the use of 
a lag window. A criterion of goodness must be defined and its effect on 
the accuracy of the spectral estimate must be determined. It is entirely 
possible that an acceptable rational for smoothing or extrapolation may 


lead to a very unacceptable spectral estimate. 
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Generally speaking, two different viewpoints have been adopted con- 
cerning the "best" way to achieve smoothing and extrapolation. The first 
adopts a purely mathematical viewpoint and proceeds on the basis of well 
established numerical techniques for fitting numerical data with exact 
functions. This approach is often called a parameter free procedure (Velt- 
man et al., 1972) because it does not depend upon any a priori assumptions 
about the observed process or the resulting data. The resulting spectral 
estimate is independent of any model for the process. The second viewpoint 
insists that an estimate of the "true" autocovariance function should be 
developed by assuming a reasonable and acceptable model for the process 
or the data. 

The parameter free procedure for smoothing and extrapolation is not 
well-suited for spectral analysis applications because it does not allow 
for the input of important information that ‘a be known about the process. 
Also, numerical analysis techniques employ criteria of goodness that can- 
not be easily related to the determination of the eaiaey of the spectrum. 
In practical applications, the addition of known information about the 
process can be used as a powerful tool to improve the spectral estimate. 
For these reasons, it is usually better to choose a method based on some 
model. 

Estimation procedures that employ some type of process model do not 
have to be as restrictive as one might imagine. It usually turns out that, 
for purposes of general spectral analysis applications, the process model 
is designed to have wide applicability and only very modest restrictions 


on the data function are. needed. If a considerable amount of model infor- 
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mation is known about the process, the user is well-advised to design a 
spectral estimator that incorporates all of this information. This un- 
usual situation precludes the use of a "standard" procedure. 

Process models that have been very successful for obtaining smoothed 
and extrapolated autocovariance functions are those used in autoregressive 
spectral estimation and maximum entropy spectral estimation. These tech- 
niques operate directly on the observed data function to produce both the 
estimated autoregressive coefficients and the estimated autocovariance 
function. The power spectrum can then be estimated by using either the 
autoregressive spectral estimator (Chapter XIV) or by taking the Fourier 
transform of the estimated autocovariance function. 

For the autoregressive estimator, it is assumed that the observed 
data can be modeled by an mth order autoregressive series. Once the 
series coefficients have been determined by some appropriate smoothing 
and fitting technique, they are used in the spectral estimator. In the 
maximm entropy method, it is assumed that the process produces a spec- 
trum that has maximm spectral entropy subject to the constraints that 
the process can be modeled by an autoregressive series and that the esti-: 
mated autocovariance and the estimated power spectral density form an 
exact transform pair (no window function). 

The estimated autocorrelation function produced by these methods 
satisfies all of the theoretical requirements set forth in the discussion 
on frequency domain analysis in Chapter IV. It is discrete but it obeys 
a well-defined mathematical equation and is defined for all lag values. 


This estimate can be considerably different from the one obtained using 
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the Blackman-Tukey method. It is now generally accepted that the maximm 
entropy method of spectral analysis has a spectral resolution superior to 
that of the Blackman-Tukey method . 

It is difficult to see just how these techniques develop a smoothed 
and extrapolated estimate unless the autoregreséive series approach to the 
characterization of a random time series is studied in considerable detail. 
This is not necessary for our purposes and the interested reader is re- 
ferred to a book by Koopmans (1974). It is important for the reader to 
realize that these techniques can be described by the mathematics associ- 
ated with the recursive digital filter. For example, the maximm entropy 
algorithm can be described as a method for the design of a filter that 
prewhitens the input spectrum. The filter coefficients can then be used 
to obtain a spectral estimate (Ulrych, 1972b). This is also commonly re- 
ferred to as prediction-error filtering (Peacock and Treitel, 1969). The 
Burg aigorithm for determining the sea ios Gants spectrum is a recursive 
procedure for calculating the prediction-error filter and the error power 
of the prediction (Smylie et al., 1973). 

The maximum entropy method, in particular, assumes that the process 
can be accurately described by an all-pole filter model (a "purely recur- 
sive" digital filter) and that the error power is minimized by a least- 
Square-error criterion. These assumptions also apply to the smoothed and 
extrapolated estimate of the autocovariance function. 

An excellent discussion of smoothing and extrapolation techniques for 

estimating witocoxeeiation functions with several examples is given in 2 


paper by Veltman et al. (1972). 
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A third method of developing a spectral estimator involves the use 
of recursive digital filtering. It was stated in the previous discussion 
that a recursive digital filter could be used to obtain a maximm entropy 
spectral estimate by using the concept of prewhitening. It can also be 
used in the more classical sense as a digital filter operating directly on 
the observed data. 

An analog spectrum analyzer sweeps a narrowband filter through the 
spectrum of the incoming signal and the detected power output of the filter 
is displayed as a function of frequency. For digital spectral analysis, 
the digital filter mist be "stepped" through the desired frequency range 
and the "power output" determined at each step. At each new frequency step, 
the filter coefficients must be changed and the data processed through the 
filter. The method operates as though the actual time function were passed 
through an electrical filter and the resulting output detected to determine 
the average spectral power over the passband of the filter. Adopting this 
viewpoint will help the user to understand that the digital filter has all 
of the "reali" problems associated with the analog filter. 

There are several problems associated with the use of digital filter- 
ing for spectral analysis. The transient response or "ringing" of the 
filter and its finite bandwidth reduce the accuracy of the spectral esti- 
mate. The transient response is especially troublesome if the data record 
is so short that the filter never reaches "steady state", The required 
modification of the filter coefficients for each frequency step and the 
repeated running of the data through the stepped filter may require large 


amounts of computation time. Finally, the user must develop a filter 
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and a criterion of goodness that is suitable for his purpose and that pro- 
duces an acceptabie estimator. 

This last task is probably the most difficult one facing the potential 
user. If he is forced to use one of several "canned" procedures, he loses 
the ability to incorporate known information about the process. This is 
a serious disadvantage if model information is reasonably accurate. Unless 
a digital filtering technique has obvious advantages for a particular 
process model, it is usually advisable for the user to select another 
method of analysis. A discussion of the application of digital filtering 


to the estimation of power spectra is given in Otnes and Enochson (1972), 
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X. HILBERT SPACE VECTOR FORMULATION OF 
DIGITAL FOURIER TRANSFORM ANALYSIS 

When this research was started and the mathematical forms for Fourier 
series and Fourier analysis were being investigated, the author noted the 
mathematical similarity between the Fourier analysis equations and those of 
Hilbert space vector analysis. Having just completed a quantum mechanics 
course, it seemed worthwhile to use the now familiar Hilbert space vector 
analysis to investigate the properties of digital Fourier analysis. The 
vector formulation that will follow was worked out in detail by the author 
and proved to be extremely useful in some of the subsequent mathematical 
derivations. Since that time, the author has discovered other references 
to this Hilbert space - Fourier analysis analogy but has not seen the 
mathematical details presented. The material that will now be presented 
was subsequently used to derive the spectral mixing formula and to describe 
many of the properties of digital Fourier analysis. 


The complex representation of a Fourier series is written as 


. 27 


7 +o 7 
f(x) = Ce (10.1) 


which has complex amplitudes defined by: 


2n 1 ft/2 -5 FP mm 
c= cin) =2 e £(x) dx (10.2) 


” =-T/2 
The function f(x) is periodic with a Fourier period of T. The independent 
variable, x, ranges over + but the principal range of x is +T/2. Since 
£(x) is periodic it is completely specified by its values over the principal 


range of x. Since f(x) is not bandlimited, it may theoretically take an 
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infinite number of complex amplitudes to describe it. 
In Hilbert space vector notation, the complex representation of a 


Fourier series is written as: 


40 +0 
£(x,T) = = a 0, %>T) = 2 (a »f)a (xT) (10.3) 


n=- n=-0 
The orthonormal basis set of eigenvectors is given by 


+5 Dox 


o(x.T)=e * (10.4) 
where the vector "direction" is specified by the parameter n/T. The 


Hilbert space vector £(x,T) is an infinite linear combination of eigen- 


vectors a (x,T). The complex expansion coefficients which are the eigen- 


values of £(x,T) are defined by the vector inner product as: 


; 1 47/2 , 
a= (o,f) =F i a (x) £(x)dx (10.5) 


The "direction" of the Hilbert space vector is analogous to the Fourier 
period and the complex expansion coefficients are analogous to the Fourier 
complex amplitudes. 

The vector norm for Hilbert space vectors is given by: 

(0) =F pr? ay ecarac - = |e? (10.6) 
-T/2 —--) 

The derivation of the complex coefficients from the definition of the 
vector inner product will now be done because it is important for later 
work. The vector inner product given in Equation 10.5 is written with 
Equation 10.3 substituted for f(x,T) as. 


47/2 


I sty “a Cx)L oy »£)oy, (x) Jdx | (10.7) 
n T “1/2 ps a MH 
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where the dummy index k has replaced n. Interchanging summation and inte- 
gration and grouping the variables in x, we obtain the expression: 
+o 1 +E/(2 4 
Te = Gf) F I or (x)ay, (x) dx (10.8) 
The integral in (10.8) represents the vector inner product of the eigen- 
vectors a, and oH but since these form an orthonormal basis set, their 


inner product is the Kronecker delta function. These steps are summarized 


as: 
+o +o 
a pote a = ee (a, f)8 (10.9) 


The Kronecker deita operating on the inner product (a) contracts to 


give the inner product (a >£) and this was the original definition of the 


complex coefficients: 
I= (a, »f) =a (10.10) 


The derivation of the vector norm is shown in the same way by starting 
with the definition given in (10.6). Equation 10.3 is substituted for 


£(x,T) to give: 


1 4+T/2 +0 .* mi +o ; 
(£,£) =F di potal%y®) OC) F(a, Hey, (addx (10.11) 


Grouping variables in x and interchanging summation and integration gives: 
+o 2. Ae 1 fle, 
(0) = Eye Eof 7 i of (x)a, (x)ax (10.12) 
As before, the integral defines the vector inner product (or, +m) which 


reduces to the Kronecker delta to give: 
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+e eee = * 
(f,) = _& ye) Ps »£) Gass eo (a, »£) (10.13) 


The vector inner products, (a2f) define the complex amplitude coefficients, 


ai> and the final result becomes: 


+o +2 
(f= Caa-= £ fa [* 10.14 
> = igs Se n = n ( : ) 
es n=-e 


These same equations which have been derived in Hilbert space notation 
will also be derived using the notation for Fourier series given in 
Appendix II. The physical and mathematical significance of these various 
quantities as related both to the Fourier series and Hilbert space vectors 
will be presented. 

A periodic function can be represented by a series expansion around 
some vector basis set. The complex representation of a Fourier series is 


one such expansion that can be written in the form: 


4<o +o +4 Dx 


£(x,T) = ¢£ (a, »f)a (x,T) = ¢F ce (10.15) 
n=-® =_ 


The orthonormal basis set of eigenvectors for a Fourier series expansion 
is defined by Equation 10.4. The complex expansion coefficients, a.» are 
represented by the Fourier complex amplitudes, c. and both are defined by 
the vector inner product, (asf). 

The vector inner product can be used to calculate the complex Fourier 
amplitudes and also to simplify mathematical derivations involving Fourier 
series. It is this latter function that the author found very useful. To 
show that the complex amplitudes can be derived from the vector inner 


product of the eigenvector operating on the state function, we evaluate 
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the following integral: 


1 +0 -j 4p mx +o +5 
ref. [ zee 7 Tex (10.16) 
42, 90/2 -5 Ex 45 he 
= Eas e e dx (10.17) 


ewe FT onze 
+c 1 +T/2 +540 (k-n) 
= = c. = e dx (10.18) 
T ee/ 


k=-a 


2 


The integral in (10.18) is recognized as one form of the Kronecker delta 
function discussed in Appendix II, Equation A2.33. Replacing the integral 
with the Kronecker delta function, the contraction of Son and c. gives the 
complex amplitudes: 
co 
(a »f) = as = x cb. Hue (10.19) 
n=-0 
These complex amplitudes are used to reconstruct the original time function, 
compute the average power, and determine the discrete amplitude spectrum. 
The vector norm is used to determine the mean-square voltage or 
"average" power in the periodic time function. The vector norm is also an 
alternate representation of Parsaval's theorem. The derivation of the 


vector norm for a Fourier series is summarized below: 


; 1 ft/2 4, ~j Tox +o +5 Phx 
(5 <5] Dee Eee dx (10.20) 

-T/2 n= k=-0 
, 42, 30/2 -j)22 om +5 Fx 

= Ee terf e e dx (10.21) 

n=-@ 7 (eo T/2 

+o +00 +0 

= Ee Eq = Eee (10.22) 
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This expression for the vector norm can also be written in the more usual 


form: 


oye a2 
(££) = Efe] (10.23) 
n==-8 


Parseval's Theorem for Fourier transformable functions is given by 


(Cooper, 1971, p. 133): 


2a 


OD 


soe) ae ee) 
f x'(eyat = = f [xCw) [Paw (10.24) 
=T 


For a Fourier series it is (Thomas, 1969, p. 600): 


oS +T/2 
Fay t EGl+bly =F fA ceaee ce,£) (10.25) 
n=1 ~T/2 


In Equation 10.25 the sin-cos expansion coefficients are used. If these 
are replaced by the complex coefficients, this expression is equivalent 
to the vector norm equation. To show this we expand the vector norm 


equation using (10.6) to give 


2 We 
f(x)dx = |e | (10.26) 
2 , 


o* 
where it is assumed that f (x) = f(x). 
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XI. SPECTRAL MIXING FORMULA 


A bandlimited time function, x(t), with period qT, can be exactly 


specified for all time by a Fourier series 


. 27 
+M +j T. nt 
x(t) = ZS ce “2 t< +o 
n 
=-M 
with complex amplitudes: 
1 tt/2 er 
Ae T x(t)e dt —MsnsM 
F -T,,/2 


The following quantities are defined: 


i) 
u 


Fourier period. 
2a 


nf 
n 


F 
frequency component in x(t). 


The average or “dc" value of x(t). 


ce] 
if 


Q 
tt 


ponent corresponding to Tub,» 


Mu, = The highest frequency component in x(t). 


The period of x(t) which will be referred to as the 


(11.1) 


(11.2) 


TT the Fourier frequency which is also the lowest possible 


The complex spectral amplitude of the ath frequency com- 


The complex spectral amplitudes are specified by the integral of 


Equation 11.2 but this representation can only be used to develop the 


theory or when x(t) is specified as a mathematical fimction. For empirical 


data, the integral must be approximated numerically. Suppose we define a 


hypothetical experiment where x(t) is sampled at a rate exactly equal to 


the Nyquist rate and we assume that the total number of samples taken is 


sufficient to completely determine the complex spectral amplitudes, coe 
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For these conditions, the sampling interval is defined as at=T,/ (2M +1) 
and the total mumber of samples is 2M+1. This means that the function 
has been sampled for exactly one Fourier period. | 
The complex amplitudes will be estimated by using a Riemann sum ap- 
proximation for the integral of Equation 11.2. This approximation can be 


written as follows (Appendix I, Part A): 


27 
-j— nkAt 
A 1 ™ . : tT : 
i = T At = x(kAtje (11.3) 
F k=-M 


Since the total number of samples taken is 2M+1, there are a total of 
2M+1 segments to be summed. If the sampling interval is replaced by, 


at=T,/ (2M+1), the Riemann approximation can be written as: 


. 277 
aM a 
Ex(katye 2MtI (11.4) 


Does this Riemann sum apprasineeted eve estimating the complex. amplitudes 
give accdtate estimates for these sampling conditions? This answer can be 
obtained by computing the vector inner product using the estimate. The 


The procedure is the same as that used to derive Equation 10.19. Substi- 


tuting Equation 11.1 for x(kAt) into (11.4) gives: 


. 277 2m 
| Mo-i5R ok OM Hi Lk 
. I = Mel Zoe = ce (11.5) 
k=-M L=-M 


This expression is modified by interchanging summation and combining the 


exponents to give: 


. 27 
‘ + 1 Mi titgg mk 
Ld ta (11.6) 
L=-M k=-M 


or by applying trignometric substitutions, the alternate form is: 
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2 
2M+1 


Mi 
re (L-n)k]} (11.7) 


b> 
I 


(—-ome+2 ¢ 
———|1+2 SXeos 
n t=-mq & 2Mt+1 kel 


It has been determined that the quantities in the brackets represent the 
Kronecker delta function (Appendix II, Equations A2.36 and A2.37). To 
prove this, a computer program was written for the expression in brackets 
and this expression was evaluated for various combinations of the integers 
Landn. The double precision program showed that for integers restricted 
to (L-n) S2M, the representation is exact. This discovery was used to 
eonclude that: | 


™ 


A 
I= 2Ye.8 =c (11.8) 
n L=-M L La n 


In other words, when the time function is sampled in this very special way, 
the Riemann sum estimation for the integral gives the exact values of the 
complex spectral amplitudes. Hence, the digitaily computed complex ampli- 
tudes can be used in the series of Equation 11.1 to exactly reproduce the 
original continuous time Sanction: 
Since tae reprod:ction is exact for the continuous function, it is 

also exact for the sampled function, x(kAt). If Equation 11.4 is used as 
a definition for the complex amplitudes and Equation 11.1 is written ina 


discrete form, these two equations become a basis for the definition of a 


digital Fourier transform pair: 


eee res on 
x(kAt) = Lee " (11.9) 
n=-M 
+M -j 7D nk 


1) oo M+ 1 
ec =r> ss Zx(katje (11.10) 
n M+l1 k=-M 


117 


The digital Fourier series and the digital Fourier transform (Bergland, 
1969) have the same mathematical form. From the data processing viewpoint, 
' they are equivalent. 

Discrete Fourier transform (DFT) routines including the fast Fourier 
transform (FFT) are designed to produce M independent frequency components 
and a "dc" term from 2M+1 samples of a data function. If the ideal 
sampling conditions previously described are achieved, the DFT will give 
exact values for the co If these conditions are not achieved, it will give 
only an estimate. When the discrete transform is arbitrarily applied to 
2M+1 data values, the resulting discrete amplitude spectrum will be such 
that the data values can be exactly reproduced. For this reason, the 
transform pair given above is called an exact transform pair. In general 
an estimator for the complex amplitudes will not produce a series that will 
give back the exact data values. 

A mathematical examination of the effect of sampling on the estimation 
of a discrete amplitude spectrum has led to the development of the sathaets. 
spectral mixing formula. In this derivation, the vector inner product was 
used to define the complex amplitudes and a Riemann sum was used to approx- 
imate the integral in (11.2). The goal of this analysis was to determine 
the effect of sampling interval and number of samples on the accuracy of 
the estimate. 

Suppose a time function which can be exactly specified by Equations 
11.1 and 11.2 is sampled over an arbitrary observation interval, Tye It is 
assumed that the observer is unaware of the exact value of the Fourier 


period, Te It is also assumed that there are 2N+1 samples in the interval 
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Ty and that the arbitrary sampling interval is At. 
First, we define an estimated amplitude spectral function as the dig- 


ital Fourier transform of the time series {x(kat): k=0,+1,+2,..., +N}: 


£(w) = At Ms x(keatye” JukAt (11.11) 
. k=-N 
It should be noted that this estimate produces a continuous function of w 
despite the fact that x(kAt) is discrete, The continuous nature of x( w) is 
a result of the definition of the estimation procedure. In some cases, 
this continuous estimate can be exploited to produce better results than a 
discrete estimate. 


Second, the compiex spectral amplitudes associated with the time series 


above are defined to be: 


a | +N =3 Fe nkat 
c == At £ x(kat)e N 
Bay =-N 
«20 
an ~ jo ak 
=sr1 = x(katye | N+ (11.12) 
ke-N 


These complex amplitudes are related to the estimated amplitude spectral 


function for x(kAt) as follows: 


X(2 a) Ty = (2N+ Lat (11.13) 


In other words, e. is obtained from X(u) by sampling at intervals corres- 
ponding to = 2mm/T, and dividing by the observation interval. Before 
continuing with the derivation, the following observations about the esti- 


mation process should be made: 
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1. Under very special circumstances the és are sesaniy equal to the 
“true” complex amplitudes, c. and hence can be used to generate 
the continuous time function x(t) from the Samples x(kAt). 

2. For any given data set {x(kat): k=0,+1,+2,..., +N}, the esti- 


: 4 P ‘ P : 
mated coefficients c_ specify a Fourier series given by 


n 
‘ +N : +32 nt 
x(t) = =£ ce N (11.14) 


n=-N 

which is periodic with period Ty and which exactly reproduces the 
‘ sample values x(kAt). From this we realize that x(t) and x(t) are 
equal at the times t=kjt but, in general, are unequal everywhere 
else in the time domain. For this reason it is concluded that 
there are an infinite number of x(t) functions which will fit a 
discrete data set. Any particular x(t) is completely specified 
by the sampling interval At and the total number of samples, 
2N+1. The desired goal of the estimation scheme is to obtain 
x(t) and its amplitude spectral estimate as close as possible to 
x(t) and F{x(t)}. 

3. For 2N+1 samples of x(t) there are 2N+1 values of i computed 
in a digital Fourier analysis scheme. In the "true" spectrum 
there are 2M+1 coefficients. This means that for oversampling | 
there may be many more estimated coefficients than "true" co- 


efficients while undersampling produces just the opposite. 


We now proceed to derive the spectral mixing formula. For. conven- 


ience, a continuous complex spectral amplitude estimate will be defined as: 
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ecu) = NFA = x(kat)e J UAt (11.15) 


where at=T/ (2N+1). This continuous function gives the discrete coef- 
ficients when sampled at intervals corresponding to niw= te n. 


To determine the accuracy of this spectral estimator it is convenient 


iD 
1p 
A 
tf 
iE 
it 
ip 


to compare the "true" complex amplitudes with th s This may be 
done by evaluating the vector inner product using the estimator. Substi- 
tuting the "true" Fourier series given by (11.1) into Equation 11.15 gives 


the following estimate in terms of the "true" amplitude coefficients: 


2 ‘ 
WM +i LAE oa, 
&w = eat ‘= Ege Fog J ukat (11.16) 


This equation can be manipulated into the form: 


Ty 
WN +jk(L -”)>——. 
dca) = ee Tt, = eae ava ks (11.17) 


Using trigonometric substitutions it can also be written as: 


A 4 Wy 
C(w) = ee OF: yy fi+2 LE con(tay-abe warts (11.18) 
ze, é 
= Ec, 6(Lu.-a) (11.19) 
fear e a , 


Equation 11.18 is one form of the spectral mixing formila. In this 


form, it describes the mixing effect of sampling on the continuous complex 
spectral amplitude estimate. In another form it will be used to describe 
the spectral mixing effect for the discrete spectral amplitudes, es The 


quantity in brackets has a very special significance and will be represented 
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by §(Lu,-u)- As far as the author is aware, this concept has not been 
previously developed. It will be called the estimated Kronecker delta 
function because when it is used to describe the discrete case, its func- 
tional behavior is similar to a Kronecker delta function. It could also 
reasonably be called a Dirac delta function except its amplitude is nor- 
malized to unity and it is primarily used in estimating the spectral ampli- 
tudes in the discrete function, 

The estimated spectral function at any particular frequency is com- 
posec of the sum of all the "real" components after they have been weighted 
by the estimated Kronecker delta function. In many ways this weighting 
can be thought of as a convolution of the discrete spectrum with the esti- 
mated delta function, This viewpoint suggests an analogy between this 
‘process and the convolution with a window function as encountered in the 
Blackman-Tukey method. The concept of an estimated Kronecker delta func- 
tion can be applied to other spectral estimators. The one derived here is 
valid only for the Riemann sum approximation. 

The "filtering" effect of the estimated Kronecker delta function is 
shown with the help of Figure 11-1. The delta function becomes "peaked" 
about a frequency corresponding to w= Lay. This weighting tends to "sift 
out" the discrete amplitude corresponding to the observed frequency, w, 
and deemphasize the other components. When the aiane of observed data 
becomes large, the estimated Kronecker delta function converges to the 
“true” function. When the hypothetical sampling circumstances described 


earlier are achieved, the estimate becomes exact. 
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Fourier Frequency 
Af = 
£ 


S(Luy, - w) 


Lu, = 87 


0<ws2n (10.26) 


mY 2 4 6 8 10 12 14 16 18 20 


Harmonic Number or 
Frequency 


Figure 1l-1. A Plot of the Estimated Kronecker Delta 
Function Showing the Filtering Effect 
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The spectral mixing formula for the discrete spectral amplitudes is 
obtained by sampling the continuous complex spectral amplitude estimator 
at frequencies corresponding to akwele n. The resulting equation is: 


L=-M (11.20) 


¢ 20 - a = s&s 27k OL ln 
oa a ee co, fay +2, Fees weit. tty) 
The estimated discrete complex amplitudes are the same as those that would 
be calculated by the discrete Fourier transform. If the observation inter- 
val and the Fourier period are equal, T,=Ty> the estimate is exact. 

The discrete form of the spectral mixing formula relates the "true" 
spectral amplitudes, ci» to the estimated amplitudes, os for any given 
set of sampling conditions. The parameters of the sampler are the total 
mumber of samples, 2N+1, the observation interval, Ty» and the sampling 
interval, At= ae The parameters of the input signal are the Fourier 
period, T_, and the complex amplitudes, c. The effect of changes in the 
sampling parameters can be studied by using the mixing formula. 

.More research needs to be done to further develop the mixing formula 
into an analytical tool. Since it relates the estimated coefficients to 
the "true" coefficients, it seems possible that an algorithm can be de- 
veloped that will improve the spectral estimate by “unmixing" the mixing 
effect. This procedure would involve an optimal search for the Fourier 
period by some appropriate criterion placed on the resulting spectrum. 

Such a routine has not been worked out by the author but some pre- 
liminary ideas were developed. These ideas are generally expressed in 


Hilbert space notation and employ some assumptions that have not been 


shown to be valid. 
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The basic problem will now be formlated. Suppose an observed x(t) 
is recorded for a time Ty but it is assumed that the Fourier period is 
unknown. Using Hilbert space representations, suppose that the "direction" 
of the vector is changed until some "optimum" direction is found that gives 
as "best" estimate of the time series. This analysis would produce a 
period Tope which would approximate the Fourier period and complex coef- 
ficierts (a, »f) based upon the desired criteria. 


The state vector of the system is defined as the infinite series: 


--} ) +c 
(x,T = yt oT) gfe soe ae (11.21) 


It will be assumed that an operator, A, exists which will yield an 
“optimum" time series when operating on |. A second assumption, based 
upon intuition and the need for a suitable criterion of goodness, is that 
the "optimm" period is the one which concentrates the spectral power into 
the fewest possible spectral seonenes This a priori assumption is, in 
effect, forcing the estimate to agree with the empirical data by approx- 
imating the data with the "least complicated" spectrum. To formlate this 
idea, we normalize the complex amplitudes such that the wectse norm is 
unity: 

+o 

E le |? =1 (11.22) 

n=-0 

The entropy measure for the spectrum will be defined as: 


co 
H=- £ fe, [log|c, | (11.23) 


n= ee] 
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To produce the fewest possible spectral components, we will want to min- 
imize the entropy, H. The problem can be summarized as trying to find an 
operator, A, which operates on | to yield an "optimum" time series, 
f(x,T oot)? which is the best estimate of the observed time series. The 
, constraint for this optimization is that the entropy is minimized. Time 
did not permit the author to develop the details of this analysis or even 
to prove the basic assumptions. For the reader interested in developing 
this idea further, the author recommends using a calculus of variations 
with constraints. One example of this type of procedure has been given in 
Appendix I, Part J. 

Examples of the use of the spectral mixing formula and a further 


discussion of the use of the digital Fourier transform for the spectral 


analysis of finite time series will be presented in Chapter XII. 
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XII, FOURIER TRANSFORM SPECTRAL ESTIMATION 
A. Infinite Periodic Time Functions 


A sample record of an infinite periodic time function with period tT 


is shown in Figure 12A-1. It is assumed that the function is bandlimited 


and that it can be characterized by 2M+1 complex amplitudes. 


Figure 12A-1. A Sample Record of an Infinite 


Periodic Time Function with Period Ty 


This bandlimited function can be exactly specified for all time by a 


complex Fourier series 


iM +j a nt 
x(t)= Eee F -@St<s+o (124.1) 
n 
n=-M 
with complex amplitudes: 
1 2 /2 -j nt 
c =f x(tle Fo dt -Msns+t (124.2) 
n Tf 
F - F/2 


The following quantities are defined: 
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The period of x(t) referred to as the Fourier period. 


i) 
" 


F 

a, = tr '» the Fourier frequency, also the lowest frequency com- 
ponent in x(t). 

ei The average value of x(t). 

Ca The complex spectral amplitude or Fourier coefficient cor- 


responding to the nth frequency component, nit « 


Muy, = The highest frequency component in x(t). 


The actual number of complex amplitudes that are needed to describe a 
particular x(t) can be very deceiving. In a practical situation it may 
take hundreds of c's to characterize x(t) but many times only a few of 


them are nonzero. 


1. Fourier analysis 


The Fourier analysis of ar infinite periodic function consists of 
determining the complex amplitudes, Cc and the amplitude spectrun, X(w). 
The complex amplitudes are determined from Equation 12A.2 and the ampli- 


tude spectral function is the Fourier transform of x(t): 


| . +2 4y ; . 
F {xct)} =f ¢ ce aE I ae 


<@ n=-M 
+M +o 
2 +j(nue-w)t 
n=2 Cn Lee meee de (124.3) 


The expression in brackets is recognized as a form of the Dirac delta 


function (Appendix II, Part E) and the final result is: 


, +4 
F ixce)} = X(w) = £ ¢ 27 8(ny,-a) (124.4) 
n=-M 
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The use of a delta function allows for the existence and definition of 


the amplitude spectrum of an infinite pericdic time function. 


2. Autocorrelation function 
By applying the definition given in Appendix I for a time autocor- 
relation function and using Equation 12.1 for x(t), the time autocorrela- 


tion function for an infinite periodic time function is easily determined 


to be: 
+M +jnuwT 
Rim =- © |e |e "F (12A.5a) 
n=-M 
Or 2 
= |e, | poe lc, | cos(ku,,7) (124.5b) 


R,(7) has an average value of les? and is periodic with period Tp: 


3. Power spectral density function 


The power spectral density function for an infinite periodic time 
function is determined by taking the Fourier transform of the time auto- 


correlation function: 
+M 2 
Friz(n) }= S(w) = ale! 2r 8(nuwy, = w) (124.6) 


For this ideal condition, the amplitude spectrum and power spectrum both 
consist of 2M+1 line components of amplitude ¢ and le? respectively. 


They both give essentially the same spectral information about x(t). 


4. Total spectral power 


The total power in the spectrum of x(t) is obtained by integrating 


the power spectral density function over all frequency: 
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1L et 2 
Total Power = on i) S.(uydw= £ lc | (124.7) 
T x n 
all w =-M 


The total power is also obtained by evaluating the time autocorrelation 


function at zero lag: 


Total Power = R,(0) = & |e |* (124.8) 


5-e Vector norm 
The vector norm is used to determine the mean-square voltage or 
"average" power in the periodic time function. It is also a particular 
form of Parseval's Theorem. For the complex Fourier series it is given by 
+7_,/2 <M 
1 Fo L* 2 
Average Power = = i) x (t)x(t)dt = £ Ic | (4124.9) 
T. n 
F -T.,/2 n=-M 
F ; 
From this representation we see that the total power in the frequency 


spectrum is determined by the mean-square value of x(t) averaged over one 


Fourier period. 


6. Ideal sampling 
Ie x(t) is sampled by an ideal sampler and at a rate exceeding the 


Nyquist rate (see Chapter VI), the sampled function can be exactly recon- 


structed using the following infinite orthonormal series: 


+0 sin ante - kat) 
x(t) = 5 x(kAt) (—_Ae ____} (12.10) 
k=-0 E(t - kbe) 


This formula can be used for exact interpolation between sampled values 


provided the sampling interval is less than the Nyquist sampling interval, 
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B. Finite Periodic Time Functions 


The infinite periodic time function x(t) is now observed for a finite 


time, T,, as illustrated in Figure 12B-1. It will be assumed that the 


N 


observer is unaware of the exact value of the Fourier period, T Although 


Fr 


Figure 12B-1. Truncated Time Sample of an 
Infinite Periodic Time Function 


x(t) is infinite, a truncated time function can be defined using a rec- 


tangular observation window function such that 


R(t) = u(t)x(t) (128.1) 
where 
tsetse, th 
u(t) = { 8) otherwise (12B.2) 
or: 
x(t) t <t<t +T . 
R(t) = y 2. (128.3) 


0 otherwise 
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For purposes of presentation, the origin in Figure 12B-1 will be shifted 


so that t = -T,/2 and the sample length is symmetrical about the vertical 


axis. 


1. Fourier analysis 


Since the truncated function is absolutely integrable, 


Loaner +,,/2 
f ixcey fae = fF * fx(ey lat <@ (12B.4) 
y/2 


the amplitude spectral function exists and is obtained by taking the 


following Fourier transform: 


+7 /2 : 
4 A - 
Xu) = F{xce)} = f : x(t)e Jae (12B.5) 

-T_/2 
If x(t) in the above integral is replaced by Equation 12A.1 and the inte- 
gration performed; the resulting estimated amplitude spectral density 
becomes: 


+M sin(nu, - wT / 2 


X(w) = ee ae ee 


=-—M ON (nu, - w)T,/2 (128.6) 


As the observation interval becomes very large, Wy" % this estimate con- 


verges to the "true" amplitude spectral density given in Equation 12A.4: 


be vl 
lim X(w) = 5 c2n &(nu, - w) (128.7) 
i," =—M 


The interested reader can verify this limit by applying Equation A2.49 


given in Appendix It. 
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For the very special circumstances where the period of observation, 
T_, is an exact multiple of the Fourier period, Ty qr,» and if we are 
interested only in the amplitude spectrum sampled at multiples of the 


Fourier frequency, w= 2n/T.» we get: 


+4 
Xcew,) = 2 9 pear e) (12B.8) 


The sinc function in the brackets is actually a special form of the 
Kronecker delta function given in Appendix II. Replacing the sine function 


with the Kronecker delta and using the contraction property gives: 


+M 
facu,) = at, 2 ¢.8y <aqlpc, (128.9) 
n=-M 
From this derivation we can see that, for these special circumstances, 


the complex spectral amplitudes can be exactly computed from the samples 


of x w) by: 


c= ra ¥(nu,) | (128.10) 


This special relationship between the complex amplitudes, cis and sample 
values of the estimated amplitude spectral function, X(w), suggests a 
curious interpretation for Equation 12B.6. Except for a scale factor, Ty 
this equation is identical in form to the reconstruction equation (6D.6) 
discussed in Chapter VI. From this it is concluded that the amplitude 
spectral function for ‘ truncated continuous time function can be obtained 
by a series expansion involving the "true" complex coefficients and an 


appropriate orthornormal sinc function. 
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For a periodic time function, the amplitude spectral function can 


also be computed by applying the convolution theorem: 


#(t) = u(t) x(t) (12B. 11a) 


Xu) = UCu) *X(u) (12B.11b) 


The Fourier transform of the rectangular window function is a sinc function 
and X(w) is given by Equation 12A.4. The convolution of the sinc function 
with a series of Dirac delta functions gives the same series as Equation 
12B.6. This convolution technique cannot generally be applied if x(t) has 
any random components because X(w) may not exist. The approach to use for 


random functions is discussed in Part C of this chapter. 


2. Estimated autocorrelation function 
There are two possible definitions for the estimated time autocorrela- 
tion function, & (1). ‘The first definition is valid for lag values mich 
smaller than the observation period, TST. This definition is consis~ 
tent with the usual assumptions applied in the Blackman-Tukey ached: 
2 + 


/ 
R(1) = ee iy x (t)x(t+ t)dt <7, (128.12) 
Ty “2/2 


This definition also yields the simplest results 


A 
R(n = 2 = e 128.13 
mo) nz a Tee ( ) 
4M tju,kt 
= ° Be (128.14) 
k=-M 


where 
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F Ty 
sint (k- Le 
Ho = Ye, [(—__—_—+ J (12B.15) 
mk - L) 
F 
and 
MM 
A, ee Oe (12B.16) 
The amplitude constants, 8. in Equation 12B.14 play the same role as the 
amplitude constants, le I?, in Equation 12A.5. In this sense, the auto~ 
correlation function for the truncated time function is no more complicated 
than that for the infinite case. The very definite difference between the 
two is that the estimated autocorrelation is itself a random function. As 
the window function moves with respect to x(t), it gives different sample 
realizations and thus generates a randomness in (1). This randomness is 
a direct result of the observation and sampling process. 
If the period of observation is again an exact mltiple of the Fourier 
period (as in Equation 12B.8), the sinc function reduces to a Kronecker 


delta and the complex amplitudes reduce to 


a. = c1e Sr (128.17) 
and: 
™ . 2 
B= & yobs = le, 1? (128.18) 
L=-M 


The net result is that the estimated autocorrelation function is identical 


to the "true" function for these sampling conditions: 
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R(7) = £ le,{e = R (1) (128.19) 


The second definition for the estimated time autocorrelation function 
is the one which will converge to the "true" function and hence is not 
biased. It is also valid for all lag values, |r| <Ty This estimate is 
more complicated than the previous one and does not lend itself to an 


easy interpretation. The estimate we will work with is given by: 


+4(T, = |r) 
R(7) — *N x(t + 9/2)x(t + 1/2)at (128.20) 


- [7] 
N “4(T, - [1 
This autocorrelation estimate can be expressed in terms of the complex 
Fourier coefficients by replacing the x(t)'s with Equation 12A.1 and doing 


the integration. After many detailed algebraic manipulations we obtain 


™ +jW.nT 
R (1) = x fe I7e F 
x n 
n=-M 
mM My, + ju, (k +L) 1/2 
+ 2 Laeq (ne cS (128.21) 
kent Lay © LK 
k#L 


where: 


(Ty - |) +ju,(L-k)t 


400 = =a J . oe 
“3(T, - |r] 
- sin{s w.(L - k)(T, - Inp} (128.23) 


Z Hp-¥) Cry |) 


The function 4 CD is even and represented by two sinc functions which are 
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mirror images of each other. Attempts at further analysis of ea T) have 
not been successful. The Fourier transform of q(t) involves a sine inte- 
gral. For [ales is I CD reduces to the sinc function in Equation 


12B.15 and the two estimates agree. 


3. Estimated power spectral density function 


For the autocorrelation estimate given by Equation 12B.12, the es- 


timated power spectral density function is 


M 
g(a) = F &(} = _E. 8.276 (nu, = a) (128.24) 
(r<<t_) 2 
N 
where 
Ty 
er aI 
6 = are te am (128.25) 
Doxey *R | ty ° 
m(k - a) 
F 


and again if ty is an exact multiple of Tp» the amplitude coefficient 
i a a! = 2 
simplifies to 8, = le | e 
For the autocorrelation estimate given by Equation 12B.21, the power 
spectral density function involves the Fourier transform of 4 6D which 
has not been determined. This estimate may be written as: 
dw = fle 
6D = z le, | 2716 (nuy, - w) 


n= 


mM MY + ju, (e+ L) 1/2 
+ 0. Lee P{q (ne 3 (12B.26) 
ke-M et ol 

k#L 
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4. Total power estimate 


The total spectral power for each estimate is obtained by integrating 


the spectral density over ail frequency. This gives for |r| <tTy, 


1 caer +M 
Total Power = >— is coaw pa B (128.27) 


and for all |r| <T; 


me 
Total Power = 2 le | 


=-M 
™m IM, 1 7 +ju,(k +L) 1/2 
+ £ Lack = (Tye d 
dea eb oe ie LO, Jaw 
k4L (12B.28) 


It can be shown that the sum in Equation 123.27 is positive and real. 


5. Vector norm 

The vector norm or Parseval's Theorem for the truncated function is 
similar in form to that for the infinite periodic time function. The 
major differences are that the mean-square value is averaged over the ob- 
servation interval and not over the Fourier period and the amplitudes, 
le; are replaced by the amplitudes, Boe. The approximation to Parseval's 


Theorem becomes: 


ge + , 
af x@xc@ae= 5 Bees Ss waw (128.29) 


Ty -T,/ 2 n=-M 


6. Frequency resolution and spectral mixing 


The ability to resolve line spectra and avoid false spectral peaks is 


of great concern when doing high resolution spectral analysis of periodic 
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signals, These topics can be studied by assuming a reasonable model for 


the spectrum of a periodic signal and then doing Fourier spectral analysis 


on the model. A convenient spectrai model is shown in Figure 12B-2. The 


X Cw) 


Figure 12B-2. A Convenient Spectral Model 
for Resolution Studies 


periodic time function can be modeled by a Fourier series of four terms: 


x(t) = A, cos (27 fit +4) + A, cos (2nf, t+ $,) 
+ A, cos (2nf,t + $,) + Ay cos (2nf,t+ $4) (128.30) 


A complete Fourier series representation would involve determining integers, 


m, and a Fourier period TP such that: 


(128.31) 
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The time function would then be modeled by a Fourier series such as in 
Equation 12A.1 where many of the complex amplitudes would be zero. 

To be able to compute a Fourier series for the periodic waveform 
requires a knowledge of the Fourier period, Th If this is not known, it 
is difficult to determine the integers m> ™,> M> and My « 

The time autocorrelation function for an infinite record of x(t) is 


computed using Equation 12A.5 as: 


Nie 


a cos (2n£, 7) 


1 2 1 ,2 
+5 A; cos (2n€, 7) a5 2 


=i ,2 
RC) = 5 A, cos (2nf 1) + 


cos (2nf, T) (128.32) 


If the infinite autocorrelation function were now truncated it would be a 
straightforward matter to use a lag window function and continue the 
analysis. The data function, however, is terminated with a data window 
and this complicates the analysis considerably. The estimated autocor- 
relation function should be computed using Equation 12B.21. 

The complicated mathematical analysis of the estimated autocorrelation 
function. will not be presented. Even the simple example of a single sine- 
wave burst discussed in Part 7 of this section involves a considerable 
amount of mathematical manipulation. Further spectral analysis for this 
section will be done using the amplitude spectral function. 

The amplitude spectral fumction for the infinite time function is 


computed using Equation 12A.4. For the four term series this becomes: 


X(w) = 7A Le JPa5(w +u) + eJag(w - uw) 


+ m Age b5(y +0) + e IE (uy - uy] 
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+ A, [e*3#25(u, + w) +e Jl gu, -w) ] 
+7 A,Le* 325 (wy, +u) + oF 5(y -w] (128.33) 


We can cay that infinite spectral resolution has been achieved in Equation 
_12B.33 because of the Direc delta functions. For a finite observation 
period, the spectral resolution becomes finite. 

For mathematical convenience, the finite time function can be thought 
of as the infinite time function being observed through a finite data 
window. We write the observed function as the product of a rectangular 


window function and the infinite time function as: 
A 
x(t) = u(t)x(t) (128.34) 


The rectangular window function was defined in Equations 12B.2 and 128.11. 
‘. . 
The amplitude spectral function for x(t) can be obtained from the convolu- 


tion of the Fourier transforms of u(t) and x(t). The relationship is: 
i. 
Rw) = F=f U(w- AKCAA (128.35) 
aD : 


The Fourier transform of a rectangular window function with unity height 
and width Ty is: 


sin ay 
uw) = TE Z | (12B.36) 
aaa 


The convolution integral is evaluated using (123.33) and (12B.36) to give: 
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Ty Ty 
‘ sin (w_ +) 7 Sin (w-w) = 
X(w) ae f a He a 2 
a (w, + W) (w, - ¥) 


sin (ut sin (wy, - w) 

+ alate 
T T 

sin(w, + w) ms sin (w, - wt 
+A wy ae 7 WS 


Cw, + w) (w, > w) 
T T 
sin (w, + 5% sin (w, - w= 
+ a TT + — (128.37) 


The phase factors have all been omitted for simplicity because we are only 
interested in the amplitude of the spectral function. 

The spectral estimate above becomes exact when the observation inter- 
val goes to infinity because the sinc functions converge to Dirac delta 
functions. This is illustrated in Equation A2.49 in Appendix II. A 


typical spectral term in the estimator is represented by: 
Ty | sin (w,- w) 3° 


a2 (128.38) 


(w, - w) > 


The "spread" or width of this spectral component is characterized by the 


width between first nulls of the sinc function, 


bandwidth-between-first-nulls = Atay = r (128.39a) 


or by the bandwidth between the points where the amplitude is down by a 


factor of (2%: 


» 161 
BW. dB oT, (12B.39b) 
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If we were to do a Fourier series analysis of #(t) and we knew the 
exact Fourier period we could set the observation interval equal to tT, 
and obtain an exact answer. This criterion of observation can be applied 
to the Fourier transform to determine the spectral resolution. If Ty = Tp 
and we examine the typical frequency component at Was the period of obser- 
vation can be replaced by 


m 
ae = ra (128.40) 
a 


where m, is one of the integers discussed for Equation 12B.31. When this 


is substituted into Equation 12B.37, the result becomes: 
73 
m, sin (2m£, - 2n£) Ze, 
é =A == [————————————] (128.41) 


a 2f. m, a 
(2nf, - 2nf) 26, 


The ratio m,/ fe is multiplied through and, the sin term reduced to give: 


sin [rn m(1 -5)] 
g=aA ee (128.42) 


The amplitude of the typical spectral component is evaluated by 


taking the limit: 


ge _4a = 
Pag § = A, f = At, AT (128.43) 
a 


The bandwidth-between-first-nulls is 


2w, af. 
A®ag = a or Af = a (128.44) 


and the 3dB bandwidth is: 
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Sw. 
— (12B.45) 


BY345 ~ Om 
a 


At this point a resolution quality factor or "Q" could be defined as: 


ee ee 
Qo. = Ding 2 (123.46) 


We are now able to draw several conclusions from this example. First, 
if the Fourier period is exactly known, the Fourier series technique gives 
infinite spectral resolution. Second, the width of a typical spectral 
component in the Fourier transform is directly related to the frequency, 
f of the spectral component and the number of cycles, m> that particular 
component has been observed. The spectral bandwidth in percent of the 
observed frequency is directly related to M,» the number of cycles observed. 
Third, for a given observation period, the higher frequencies will get the 
better percentage resolution because more cycles have been observed. This 
results in a higher "Q", 

The spectral resolution for the frequency components {, and fo can be 


determined by examining the amplitude spectrum near these frequencies: 


a) 


sin Cw, = w) aa 


15 Cw, ~ w) ] 
N 
2 


x(a) ~A 
wo, 
sin Cw, - w) 


(=D ] (128.47) 


+a, [ 


Since wy and W, are nearly equal, the two spectral components have approx- 
imately the same spectral handwidth. For equal amplitude spectra we de- 


fine the Rayleigh resolution as one-half the bandwidth-between-first-nulls: 
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Rayleigh resolution = - Au, — (12B.48) 


vn ON 


Using this criterion, the two spectral peaks are just resolved. For the 
case where the observation period, Ty is equal to the Fourier period, Th» 


the Rayleigh resolution is: 


fi: if 
i ~~ 2 
5 Afey am (128.49) 


Again we see that the more cycles observed, the better the resolution. 
We will also define a Fourier resolution as equal to the bandwidth-between- 


firstenulls: 


Fourier resolution = Au = = (1238.50) 
N 


For this spectral separation we say that the spectral components are com- 
pletely resolved. 

The resolution criterion must take into account differences in spectral 
amplitudes because a large component near a small one may cbscure the 
identity of the smaller component. A criterion is difficult to establish 
for dissimilar amplitudes. The "leakage" of spectral power from the large 
component onto the frequency of a smaller component can be fairly well-de- 
fined. For a Fourier resolution criterion we will say that a component 
has been resolved if the estimated spectral amplitude is Larger than twice 
the amplitude of the leakage spectra. In terms of the example, this means 
that: 

Ty 
sin (Au..) > 
: rr 


X(w,) >2A, [ (12B.51) 
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The criteria used to select an appropriate observation interval will 
now be summarized. First, to resolve the lowest frequency component, fo 
the number of cycles to be observed depends upon the absolute frequency 
and the desired bandwidth between first nulls: 

2f. 
. # of cycles observed = m= ag (12B.52) 
The needed observation length for distinguishing the lowest spectral com- 
ponent from zero frequency is obtained by equating one-half of the band- 
width-between-first-nulls to f.° This gives, m,=4. Second, to resolve 
two closely spaced sinewaves by the Fourier resolution criterion requires 


an observation period of: 
> 2 (128.53) 
ty 7 TE : 


Third, to determine the observation period required to resolve the band- 
width, f,-f,° into m, increments of frequency resolution it is necessary 
to compute the bandwidth-between-first-nulls es the ratio of the total 
bandwidth to the total number of increments. This is used in (12B.53) to 


give: 


T,> s (128.54) 


7. Sinewave burst spectral analysis 


In somewhat of a digression from the material in the previous section, 
we consider a single frequency sinewave observed through a data window of 
width, Ty The phase of the sinewave with respect to the origin of the 


data window is arbitrary and contributes a random aspect to the sampling 


146 


observation. The observed function illustrated in Figure 12B-3 is some- 


times referred to as a sinewave burst. The infinite cosinusoid is ex- 


pressed in the phase angle representation as 


x(t) = Acos (wit + $,) (128.55) 


with amplitude spectrum, time autocorrelation function, and power spectrum 


respectively: 
43 a 

X(u) = 1 Ale ogc + w) +e Jos(w - w) J (12B. 56) 
1 ,2 

R17) = A™ cos wT (128.57) 
Tt ,2 

P(w) = FA [6(a, +4) + 8(w, - w)] (128.58) 

x(t) Data "window" 


Figure 12B-3. Sinewave Burst or Single-Frequency 
Sinewave Observed Through a Data 


Window 
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For the Fourier spectral analysis of a sinewave burst of arbitrary 
reference phase, we can model the observed function as the product of a 


rectangular window function and the infinite time function: 
H(t) = u(t)x(t) = u(t)A cos (wt+9,) | (128.59) 


The rectangular window fymction u(t) was defined in Equations 12B.2 and 
A 
12B.11. The amplitude spectrum of x(t) is obtained from the convolution 


of the Fourier transforms of u(t) and x(t). The relationship is: 
A 1 2 
x(u) = 5= f UCw- AE(A)AA (128.60) 
aT =o 


The Fourier transform of the rectangular window function is 


] (128.61) 


and the convolution integral becomes: 


° 2 -j¢ (e) 2 
Cw, + w) ¢ ws w) 


Xu) = AfetI% J (12B.62) 


This is the familiar amplitude spectral function previously presented in 
Equation 128.37. The sinewave burst may be effectively analyzed with this 
function. If one applies the autocorrelation concept to the sinewave burst 
in an attempt to obtain the power spectral density function, the mathe- 
matics becomes very complicated. 

For the sinewave burst, the estimated time autocorrelation function 


can be computed using either the definition in Equation 12B.20 and inte- 
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grating directly or by applying Equation 12B.21. For this discussion we 
will apply Equation 12B.21 to demonstrate its usefulness to the reader. 


In complex notation, the sinewave is written as 
x(t) = ce J%" + ¢ eT Ste (128.63) 
where (see Appendix II): 


a 2 ; Ae 3% oe oA et5%o (128.64) 


Applying (12B.21) we get 
2 -j 2 43 
GD) = |c_| e JT y lc, | e347 


* 
+ c.c.q (7) + C, * -4_ (7) (12B.65) 
with: 
sin w, (Ty - I7]) 
q(T) = 4,(7) = —oG- Fp (128.66) 
Simplifying terms and adding we can obtain the estimated time autocor- 
relation function for the sinewave burst as a function of both lag and 


observation period 


re 
&(1,7,) an) A” cos wT 
sinw (1, - It) 
he = A cos 26, > a= FD (128.67) 
where it must always be remembered that the lag must be less than the 
observation period, || <ty and that the autocorrelation is zero for lags 


larger than Ty 
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R,(1,2y) is really a fumction of the random variable, o5° introduced by 


the randomness of observation. The expectation value of the estimate is: 
E{R_ (1,1 ,)} = RCD |r| <t (128.68) 
x* ?°N x N . 


The last term in (12B.67) is the direct result of not having an infinite | 
record to average over time and thus represents a distortion in the auto~ 
correlation function obtained for an infinite sinewave. We now examine 
RC 1,0.) to determine the effect and seriousness of the distortion. 

The two terms of the sum in (12B.67) are plotted in Figure 12B-4, 
The most serious distortion of the ideal cosinusoidal shape of the auto- 
correlation function occurs within a distance, T> of the observation 
interval, Ty: For small observation intervals, the sinc function will 
"mask" the identity of the sinusoid. For large observation intervals, 
the effect of the added sinc function is reduced to a small percentage. 
We will look at two limiting conditions for R(t.) and compute the 
limiting effect on the amplitude estimate and the spectral estimate. For 
lag values mich smaller than the sibersatien interval and for an observa- 
tion interval much larger than the period of the cosinusoid: 

In] <n, 


£ 7,Ty) ~ AO Bes wot = RC) (128.69) 


y??T, 


This result is expected because we want the estimate to converge and be 
unbiased for large observation intervals. Next we look at the estimate 
of the total power because it is used to obtain the estimate for the 
mean-square value of the. coatausoia which is in turn used to estimate the 


amplitude, A. For zero lag, the total power estimate is: 
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sinc 


a OP 
= = 
+?) 


Figure 12B-4. Two Components for the Estimated 
Autocorrelation Function of a Sinewave 
Burst 
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sin wT 


4} (128.70) 
w N 


ae =i ,2 
Total Power = R(0,T,) => A“[1+ cos 2¢,, 


A plot of this function is shown in Figure 12B-5 to illustrate how the 


total power estimate converges as the observation interval increases. 


& (0,T,) 


Figure 12B-5. Convergence Function for the Total 


Power Estimate of a Sinewave Burst 


The maximm error in total power is 2 or a factor of 3db. The absolute 


percentage error for worst case phase and frequency is obtained by 


evaluating: 


5A UL + >t ] 
oN 


(128.71) 
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Table 12B-1 shows the observation interval as some miltiple of the period 
qT and the resulting percentage error in total power. It must be re- 
membered that these are absolute worst case values and actual results may 
be considerably better for observation periods exceeding To: Also, the 
randommess of the initial phase of the cosinusoid will have the effect of 


producing a randomness in the estimate. 


Table 12B-1. Percentage Amplitude Error as a Function of Observation Inter- 
val 


Observation Interval Percentage Error 


Ty = 1.59T, + 107 
Ty = 3.18T, + 5% 
T. =15.9T +13 
N ° = 

T. = 159T + 0.1% 
N ° - 


- A 
The Fourier transform of R(t, T,) is used to obtain an estimate of the 
power spectral density function for the sinewave burst. The power spec- 


trum can be represented by two components 
(R.(1.3,)} = $5 Cw +8, (w) (128.72) 


where the integrals over the interval + Ty become: 


ty 


< = 2 2 -jur 
S,¢(w) = 35 A i cos w Te dt (128.73) 


“Ty 
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ty sin w, (Ty - |r|) re 


~ 1 .2 
S.(w) 25 A’ cos 24, f oe -rp e dt (12B.74) 


“Ty 


The analysis could be continued using these definitions but much more in- 
sight into the processing problem can be gained by introducing a lag 
window function, D(1,T,)- A plot of the estimated autocorrelation function 
showing the application of a rectangular window function is shown in Figure 


12B-6. The estimated power spectral density function becomes 


A 
$s (w) = $ Cw) + $,¢u) (128.75) 
where: 
nh 
f me ares ~jWwT 
S,(u) = 5A i cosu te 7 dt (12B.76) 
L 


7: sin w (Ty - |r|) aaa 


gu) = > a2 cos 29, i 2 -TD e dt (12B.77) 


-T 


These estimates will converge to those in (12B.73) and (12B,74) when 
Tv. Since both parts of R (1,1) are even, the cosine transform can 
be applied to give: 

sin(w, - wT, sin(w, + wT, 


=i a Oe 
$C) me art, [ (w, - w)T, - (a, + wT 


(128.78) 
+E sin w (Ty - |r|) 


ee ee 
$5 (a) =F A™ cos 2$, f —oG,- TD cos widtT (128.79) 
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Figure 12B-6. A Rectangular Lag Window Function Applied to 
Estimated Autocorrelation Function of a Sine- 
wave Burst. 
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These two spectral components must be analyzed to determine the accuracy 
of the estimate. The g(a) component is exactly what would be obtained 
from applying a rectangular window function to an infinite autocorrelation 
function obtained from an infinite time function. In this sense, g¢ w) 
represents the "true" spectrum obtained by a finite observation. The 

| $C) component represents the additional spectrum generated because the 
time function was truncated before the autocorrelation function was 
computed. 

The expression for $,(u) given in (12B.79) could not be integrated 
directly because an analysis of the integral by a change of variables 
showed that it was a sine integral and could only be solved by numerical 
techniques. If it is assumed that the lag window width is much smaller 
than the observation interval, T, <<Ty> the spectrum represented by $(u) 


can be written as 


12 sin w, Ty ot 
SKC) = S. A’ cos 26, Tg cos wrdt (128.80) 
: fe) -T. 
T, <Ty L 
2 sin W Ty sin wt 
= A°T. cos 246 ,[————][__—— (128.81) 
L fo) wre wT, 


The power spectral density function for this special case is plotted in 
Figure 12B-7, The effect of truncated data and a rectangular data window 
on the sinewave burst power spectrum is an added sinc function term at 
zero frequency and a distortion of the total power estimation. The ampli- 


tude of the de term is a factor of 


sin w, Ty 
Oy 


2 cos 2¢,L 7 (128.82) 
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$6) = 8, (w +8, (a) 


9 sin W, an 
AT. cos29 [ 7 

2 
L ° Wot 


Figure 12B-7. Power Spectral Density Function for a 


Sinewave Burst where T << Ty 


smaller than the "true spectral amplitude cf $ Ca). For practical sit- 


uations, Ty 7?7T,° and the worst case value for this factor becomes: 


2 T, 


2=8 (128.83) 
“oN Ty 


For a dc amplitude of less than one-percent of the "true" amplitude, the 
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sinewave burst must be observed for more than 31.8 cycles. 

These derivations for a simple time function such as a sinewave burst 
help illustrate the mathematical difficulty associated with trying to 
calculate the power spectrum for a truncated periodic function. These 


problems are usuaily solved numerically. 


8. Errors in amplitude spectral estimation 


A continuous but finite record of a noiseless periodic waveform with 
Fourier period Tt is sampled for a time Ty The accuracy of the Fourier 
coefficients calculated from this sample record will depend upon the length 
of the sample and the relative amplitude of each frequency component in the 
spectrum, The Fourier transform estimate of the amplitude spectrum is 
given by Equation 12B.5. This estimate can also be written in terms of 
the Fourier frequency, the "true" complex amplitudes, and the observation 
interval as given in Equation 12B.6. 

There are actually four stages of estimation which may be considered 
in determining the accuracy of our estimate. Stage-1 is represented by 
the actual Fourier transform given by Equations 128.5 and 1238.6. These 
estimates are continuous functions and do not directly represent the com- 
plex amplitudes. Stage-2 is the discrete spectral estimate obtained from 
doing a discrete Fourier transform such as the FFT. The discrete spectrum 
obtained by this approach is represented by: 
+T,/2 -j 2 ee 


y= f MN xceye TN oat (128.84) 


Ty “1/2 


Xk 20 


or by modifying (12B.6): 
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n2n 
te? is - 4 
X(k =) i c (12B.85 
i corner nN Ge - Ty /2 ) 
Tt 


This discrete spectrum may or may not correspond to the spectral frequen- 
cies representing the "true" Fourier spectrum. If the Fourier frequency 


were known, a stage-3 estimate would be defined such that 


+1, /2 ~ju,t 

Aku.) =f  x(tde  * at (12B.86) 
“1,,/2 

ors 
M sint (n- kt 

X(ku,) = = eo. ———— + (128.87) 

n=-M N 
m(n « k)~ 


F 


Equation 128.87 can be solved to give an exact expression for the k-th 


complex amplitude as: 


Ty 
7 , vr Ser 
==X(kw) - Ec (128.88) 
“; Ny oF n=-M 2 a 


The second term ea the right-hand side of (12B.88) can be thought of as an 
‘error correction factor for the Fourier transform estimator. If the 
Fourier ee ee approximations for the complex amplitudes in the error 
correction factor are substituted into (128.88) » the stage-4 complex 


amplitude estimator is obtained: 
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ty 


Sint (n- Qe 


+M 

a on A ‘ Tc. A F 

cq. = cme X(ku,) ts ae8 X(na,) a (12B.89) 
nfk mn = 05 


This estimator is not practical unless the Fourier frequency is known or 
can be accurately approximated. 

The unknown parameters for the estimation problem are Tt and c 
These must be determined from x(t) by using Ty and Equations 123.5 or 
12B.84. The error associated with the amplitude spectral estimate is 
mostly dependent on the observation time, Ty Equation 12B.10 showed that, 
if Ty was an exact multiple of Tp the complex amplituded could be computed 


exactly. When Ty is not an exact multiple, we can use Equation 12B.87 to 


estimate the error. First let 


Ty = at, + at (128.90) 


where: q=i1,2,3,.-. and 0<AT<T,. 
This is substituted into 123.87 and after many manipulations we obtain: 
+M - 
e. ia At rsinn (n-k)e¢ 
Xckuy,) ae cat, cos m(n-=-k) T. [ mae 1 
i ~~ 4t 
+ sint(n kt 


; 
+ an cat, cos T(n-k)q[ mn-k)q J (128.91) 


This can be further reduced to: 
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™ 


X(kw,) = qi, = c cos n(n = k)Se [8 ed 
n=-M F 
aM T 
+ AT Y ec cosn(n-k)q sinc[(n- KAty (128.92) 
=m 2 Th 


The first term on the right-hand side of (12B.92) is reduced to qT oy by 
using the sifting property of the delta function. The second term is an 
error term caused by not sampling at an exact multiple of the Fourier 
period. The principal error term is AT cy and is found by letting n=k. 


The estimate of (12B.93) is now written as: 


ae! T 
¥(ku,) 2 (qT,.+ AT), +AT OL ¢ cos m(n-k)q sinc[(n- KA] 
n=-M F 
n¢k 


(128.93) 


This equation reduces to Equation 12B.9 when AT~0O. As the number of 


periods of observation increases, the term qt, dominates and the estimate 


converges. 


C. Finite Random Time Functions 


A sample time function for a random process exists for all time, 
-7Sts+, A truncated time function is defined from this sample by 


multiplying it with a rectangular window function such that 


x(t) = up (t)x(t) (12¢.1) 


where u,(t) was defined in Equation 12B.2. For this condition, x(t) is 


absolutely integrable and an amplitude spectrum exists and is defined as: 
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Se +T/2 
XC) =f (rye Jae = f x(tje Jat T<o (12C.2) 
‘=£/2 


=o = 
One might expect to apply the convolution theorem to obtain Cw) but this 
requires the existence of the Fourier transform of the infinite time 
Sueetoni, x(t). For random processes, this transform generally does not 
exist. 

This conceptual dilemma can be resolved through the use of Parseval's 
theorem. The power spectral density function sought is the distribution 
of average power over frequency so for the previous estimations this is: 

+co co 
z I xp (t)dt = 2 Pec ee T<e (120.3) 
The term on the left is an estimate of the mean-square value of %p(t) 
averaged over a time, T. Both this estimate and XCa) are random because 
they are derived from only one sazple function of the process and hence 
are not ensemble averages. 

These estimates of average power can be used to derive the power 
spectral density estimate by first taking the expected value of both sides 
of (12C.3) and then taking the limit as T-°2: 


+T/2 


lind f Ef ce jae =f dim 2 efit cap | 234 (12¢.4) 
teot ty Tt 2 eee Baas iz S 


We know, for a stationary process, that the total power is determined by: 
2 ees 
E{x"(t)} = $= J 8, (adaw = R,(0) (12¢.5) 


The expectation of x(t) can be reduced as follows: 


eix2ct)} = une) Efx?(t) } = 17 (EDR, (0) (12C. 6) 


This can be used to reduce the left-hand side of Equation 12C.4) and from 


identification with (12C.5) we can conclude that: 
es 2 
5,() = ims Et ff. caw 1°} (120.7). 


This gives the power spectral density function in terms of the estimate 
of the amplitude spectrum. 

To show the relationship of this estimate to the autocorrelation 
function definition of spectral power, the Fourier fnaneesas relationship 


for ¥ Cw) is substituted into (12C.7) to give: 


+T/2 +jut, +1/2 jut, 
S_(w) a : E{ _ ata dt, Loe dt, } (12¢.8) 
+2/2 +T/2 ‘ 
-ju(t, - t,) 
= lim F et f a | ele atede 2 “Vatidt,} (120.9) 
~T/2 = 


This double integral is further simplified by using the variable substitu- 


tion, T= ty = ti> and then, ty =t: 
. +1/2-t +7T/2 


ee j 
S_(w) = lim = E{x(t)x(t+ 7) Jeo atar (12¢.10) 
ae lies t Le | 


For a stationary process, the expected value inside the integral is the 
time autocorrelation function R(1) and since it is independent of reference 


time, t, the integral equation reduces to: 


4T-t 
S(w) = lim J oR (me “JOT, 
T7o «Tet 


aa =jwr 
= f[RCne I a7 (12c.11) 
oD 
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Equation 12¢.11 shows that the estimate for power spectral density given 
by (12C.7) converges and is unbiased for large observation times. 

All of the estimation techniques which have been developed for ampli- 
tude spectral analysis can be modified and applied to finite random time 
functions by using Equation 12C.7. An estimator for power spectral density 


for an observation time T would be: ~ 


3) = SEC RCo 173 (12¢. 12) 


D. Finite Periodic Time Series 


The previous analysis of periodic time functions and random time 
functions could be applied to discrete time functions by "digitizing" all 
of the continuous equations and functions. This approach can be taken 
when the collected data is continuous and the desired degree of numerical 
resolution may be chosen. This technique is equivalent to a numerical 
solution for the sone tabas case. 

Another approach to the analysis of discrete data is to treat the 
data directly as though it is a time series. The author prefers this 
approach because it results in the use of superior data processing tech- 
niques. Data in the time domain will be treated in discrete form even 
though it would be possible to generate a continuous function using the 
sampling theorem of Equation 6D.6. 

A periodic time function, x(t), is sampled for a finite observation 
time, Ty with a sampling interval, At. This process will generate a 


time series x(kAt). The sampled function is shown in Figure 12D-1 and 
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we observe the following definitions: 


At = The sampling interval. 


1 ‘ 
f, AC? The sampling frequency. 


2N+1 = The number of samples of x(t) in the interval Ty 


The length of the observation interval. 


bl 
i] 


T_ = The Fourier period of x(t). 

Muy, =M =r, The highest frequency component in x(t). 
F 

f= ; f° The folding frequency. 


ami, = 2Muy, » aN is the Nyquist frequency. 


x(t) 


y 


Figure 12D-1. A Sample Record of a Finite Periodic Time Series 


It will be assumed that x(t) is sampled at a rate greater than or equal to 
the Nyquist rate so that aliasing is prevented. Also, the number of 


samples will be assumed to be odd so that; (2N+ 1)At=T,. 
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1. Fourier analysis 


The discrete Fourier transform has been briefly discussed in Chapter 
XI and Appendix II. The discrete transform pair in the time domain can be 


represented by: 
2u 


1 WwW on Tp mise 
x(kAt) = x= X(n Te €12D.1) 
F n=-N F 
-j 2T neat 
2. nN TE 
X(n T = At & x(kAtje (12D.2) 
F n=-N 


If the periodic function is sampled such that, At=T)/ (2N+1), the discrete 


transform pair becomes 


2 
+N +j nk 
m= OF xe aN+1 (120.3) 
n=-N F 
. 27 
1. 1 Wo iI NST _ 


where the arguments kAt and n tr have been replaced by the indices k and n. 
Most discussions of the discrete Fourier transform actually use the 
complex Fourier coefficients, ci» as the amplitude spectral component when 
actually it should be x: This is done in Bergland (1969) and in Equations 
11.9 and 11.10 of this text. As a practical matter it makes little dif- 


ference as they differ only by the scale factor T This can be seen by 


= 
comparing Equations 11.9 and 11.10 with 12D.3 and 12D.4. For a given time 
series {x,: k=0,+1,+2,...,+N}, the comp lex amplitudes computed by 


(12D.4) represent those which are needed to exactly reproduce the original 
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time series using Equation 12D.3. This feature of the discrete Fourier 
transform is discussed in Chapter XI. In the discussions to follow, we 
will use the circumflex accent over x, and x to denote that they are 
really estimates and the exact Fourier period, T, » will be replaced by the 
observation period, Ty Also, in most cases we will work with the complex 
Fourier amplitudes, c_ = x / Ty 7 

The theory behind the discrete Fourier analysis of a finite time 
series must be studied to determine the interpretation of the estimated 
spectrum. Given a finite time series {x(kAt): k=0, +1, +2, ..., +N} we 
have 2N+ 1 ordered data pairs tx, sk}. This data set can be used to define 


2N+1 complex Fourier coefficients according to the following algorithm: 


. 27 
ae ee 
é =o Se Coe _ (12p.5) 
® k=-N 


These complex coefficients are actually approximations to the "true" co- 
efficients representing x(t). These estimates are then used to define a 


bandlimited Fourier series of the form: 


" . 27m 
+5 —t 
rN +N A ; Ty 
x(t) = x ee T.=(2N+1)At (12D. 6) 
n=<N 2 N 


This estimated Fourier series has the following properties: 


1. x(t) is continuous and periodic with period, Ty (2N+1L)At. 
2a 


Ty 


2. The highest frequency component present in x(t) is, anf, =N 
N 
2N+1 zo 


3. The average value of 2(t) is ¢ 


2n 
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4. The mean-square value of x(t) is computed by: 


+N 
ee Fr le (7 (12D.7a) 
n 
=-N 
5. The critical frequencies of the discrete transform are: 
fo =>=—2—Ff (n=41,42 +N) (12D.7b) 
B.D. (2NEL SOP erie eS : 


r A 
The most important feature of x(t) is the fact that the values of x(t) at 


t=kAt are exactly equal to the data set values x(kAt), in other words: 
A 
x(kat) =x(kAt), k=0,+1,+2, ..., iN (12D.8) 


The complex spectral amplitudes in Equation 12D.5 represent the spectrum 
of x(t) exactly while x(t) approximates x(t). It seems reasonable to 
assume that if enough samples of x(t) are taken so that it is very closely 
approximated by x(t), the spectral estimate, es will approximate the true 
spectrum, c- 

The continuous function, x(t), generated from the c by Equation 
12D.6 and hence from the finite time series {x(kAt): k=0, +1, +2, ...4N} 
can be said to be "optimum" in the sense that x(t) at t=kAt exactly 
reproduces the original tiie series with a Fourier series bandlimited to 
N/T, - This bandwidth limitation provides the "smoothest" extrapolation 
between data points consistent with the restriction that ¥(t) be periodic 
with period, T,.. The fact that x(t) is both periodic and bandlimited 


N 


provides considerable mathematical convenience for analysis purposes. 
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To illustrate the discrete Fourier transform we will look at a simple 
example. PROGRAM 01 in Appendix III has been written for the example. The 
line spectrum subroutine in this example is a straightforward digital 
transform. For large amounts of data processing, the FFT should be used. 
Consider a time series obtained by sampling a continuous time function 


represented by (see Appendix II) < 
x(t) = 4 cos(2nt) + 1 sin(6mt) (12D.9a) 


where, a,=4 volts, f,=1 Hz., b3= 1 volt, and f,=3 Hz. The Fourier 


frequency is 1 Hz. and the highest frequency is 3 Hz. The complex ampli- 


tudes are calculated to be: 


Coe c, = 0 
ae 2 
e¢, = C4 = 
-j2 
Eh ed ’ ae ee 
C2 = C 3-5 e = 95 (12D.9b) 


The data inputs to the program are the a's and b's, the Fourier period, 
the period of observation, and the number of data points. The program 
generates x(t) gaugiea at the intervals At=T,/ (2N+ 1) and calculates the 
magnitude and phase of om for, n=0,1,2,...N. The magnitude of c with 
respect to a chosen reference is also given. Finally the calculated c, 
are used to generate the estimated series x(t) where x(t) is sampled four 
times as often as x(t). 

By varying tN and 2N+1 it is possible to see the effect of sampling 
on the computed discrete spectrum for the time series {x(kAt): k=0, +1, 


42.5 eee 3N}. Figure 12D-2 shows the effect of observing for less than a 


Amplitude - dB 


Figure 12D-2. 
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= 0.7, NS = 21 
0.9, NS = 21 


0.95, NS = 21 


Frequency - Hz 


Discrete Spectrum Plots Showing 
the Effect of a Varying Observation 
Period 
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Fourier period and also shows how the spectrum converges as ty approaches 
T_.. Several computer runs where Ty was an exact multiple of Ty, showed that 
the calculated spectrum was exact within computational accuracy. The 
magnitude of the complex amplitude was plotted in dB with a two volt 
reference. A dB plot is necessary because of the wide range of amplitude 
values. It is important to note that the plot for Ty = 0-7 shows no peaks. 
This may be described as a shift of spectral power toward de. 

Figure 12D-3 shows the calculated spectrum for a variety of sampling 
conditions. It is important to note that the spectral resolution of 81 
data points and Ty = 10-5 is little better than the resolution of 21 data 
points and T= 3-4. This illustrates the very important effect of obtain- 
ing and observation period as close to a multiple of Ty as possible. The 
reader may experiment with these effects by using PROGRAM 01 to study a 
variety of sampled functions and sampling conditions. For-large data sets, 
a Fast Fourier Transform subroutine should be used in place of SUBROUTINE 
LNSPTM. The high degree of spectral selectivity observed as Ty is varied 
was the reason for the discussion at the end of Chapter XI of the selection 
of an optimum observation period. This procedure has much potential in 


terms of improving the discrete Fourier transform. This theory needs to 


be worked out. 


2. Spectral mixing 


A discussion of spectral mixing and the derivation of the spectral 
mixing formula were given in Chapter XI. In this section we diseuss the 


use of the spectral mixing formula - 


Amplitude - dB 
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Figure 12D-3. Discrete Spectrum Plots Showing the 
Effects of Long Observation Times and 
Increased Number of Samples 
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A a 
ec = 2 ¢. 6 (N,T,,T,) (12D.10) 
Gq am T 14 NF 


where the estimated Kronecker delta function is defined by: 


N 
A ee 2mk rg 
bg Ne Tye Tp) ar [1+ eo cos N+ TT Ty! (12D.11) 


The estimated Kronecker delta has the following properties: 


‘ 1 r=q 

6 = (12D.12) 
4 8) r#q and Ty= Tp 

ra =O r#q and Ty >> T (12D.13) 


The ranges on the indices are: 


r= 0, +1, +2, ..., 4M 
q = 0, +1, +2, ..., +N (12D.14) 


PROGRAM 02 was written to verify the formula for the estimated 
Kronecker delta function. RUN #1 shows all values for the condition 
TF=TN=1. and M=N=5. Within a high degree of accuracy, Equation 

2D.11 is verified for the conditions specified by Equation 12D.12. For a 
large number of samples, the estimated Kroneker delta converges to the 
"true" delta function. This ‘ean be verified using PROGRAM 02 by increasing 
N and TN. The interested reader is encouraged to experiment with the 
properties of 8 og Naty ety) by using the program. 

There are a variety of ways the spectral mixing formula can be applied 
to the analysis of the spectrum of a time series. A particular application 


is needed before a specific use can be specified. One of the most useful 
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applications is the determination of the amplitude resolution limit for 


an estimated spectrum. The determination of resolution limits will be 


discussed in Chapter XIX. 


3. Spectrai resolution and accuracy 


A discrete frequency spectrum derived from digitally processing a 
time series can be viewed as a sampled frequency spectrum. Because of 
sampling, the spectrum will have a limited resolution and accuracy and it 


will be finite. For a periodic time function, x(t), the maximum and min- 


imum frequencies present are: ; 
é 


_ 20 we 
Wax = EM fix tM (12D.15) 
F F 
_ 20 ade 
Soin z Sain = . (12D.16) 


For a time series with sampling interval, At, and observation interval, 
Ty (2N+1)At, the maximum and minimum frequency components detected by 


the digital transform are: 


= 20 y ¢ =-N_L 

wax z max 2N¢+L At (12D.17) 
_ 20 a ey 

ae T ae 7 (12D.18) 


The digital transform can be viewed as generating discrete frequency "boxes" 
or frequency bands into which the computed power in the sampled signal is 
distributed. A continuous spectral function is generated from the discrete 


spectrum by expanding in a series of sinc functions (see Equation 12B.6): 
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‘ AN reall (f£- tty 
e(f)= Ze See (12D.19) 
-y 2 nm (£ tty 
The bandwidth between first nulls is: 
BWEN = 2 £ ee 12D.2 
~ W+los — Ty (12D .20) 


The center frequencies for the "boxes" of the discrete transform are 


(12D.21) 


n 
n 2N+1l At 2N+i1s Ty 


where f, is the sampling frequency. A convenient technique for concep- 


tualizing this process is illustrated with the help of Figures 12D-4 and 


12D-5 e 


4. Sampling and "sync" frequencies 


The selection of a sampling interval, and an observation period has 
a profound effect on the rae eee wnen the total number of 
samples is limited. As we discussed in the Fourier analysis section, a 
value of Ty which is exactly a multiple of the Fourier period will produce 
an exact spectrum up to a frequency of N/T. The sampling interval, 
bt=T,/ (2N+1), may determine how many unique samples of a particular 
Sinusoidal component are obtained. Consider the effect of the sampling 
interval on a single sinewave as shown in Figure 12D-6. This is effec- 
tively the same as sampling a single period of the sinewave over many 


points as shown in Figure 12D-7. 
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N=1 
(3 samples) 


N= 2 
(5 samples) 


N= 3 
(7 samples) 


Figure 12D-4. Conceptual Presentation for Illustrating 
the Distribution of Spectral Power as the 
Number of Samples Increases 
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N=12 
(25 Samples) 


Figure 12D-5. Conceptual Presentation for a 
Large Number of Samples 


This way of looking at the sampling process led to the discovery of 
unique sampling frequencies we will call "sync" frequencies. When the 
spectrum of x(t) contains a sinewave component corresponding to one of 
these sync frequencies, the number of unique samples of this spectral com- 
ponent is limited to a finite number, ase 

To determine these frequencies, consider the single sinewave in Figure 


12D-8. We can define a remainder period AL such that: 


2AL = T-nat n= 3,4,006 AL <0 (12D.22) 


177 


Figure 12D-6. Sampling a Single Sinewave 
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Figure 12D-7. Equivalent Sampling Informacion 
for One Period of a Single Sinewave 
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At 


2At 


Figure 12D-8. Single Period Sampling of a Single Sinewave 
Showing Remainder Period, AL 


The integer, mn, represents the number of increments in a period, T, before 
the cycle repeats and AL is one-half of the remaining period length. 
For, AL=0, the sync frequency and unique sampling interval are de- 


fined by: 


AL=0 At = = T f= 2 (12.23) 
For a 3 Hz sinewave and, a 7 unique samples, the sync frequency would be 
21 Hz and the sampling interval would be 1/21 seconds. It should be noted 
that any integer miltiple of a syne frequency is still a syne frequency. 
For, AL>0O, the total number of periods sampled must accumulate until 


n AL=T. The syne frequency and unique sampling interval become: 


‘AL>0 at=(¢ - +57 f = = ee 
‘2 7 n : sn n,-2 T 


| (120.24) 
B= 354,55++(2N+1) 
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For the previous example, the Sync frequency would be 42/5 Hz and the 
sampling interval would be 5/42 seconds. 

Another way to view sync frequencies is to select a given sampling 
frequency, £,, and determine where in the spectrum these frequencies will 


occur. They will occur at frequencies corresponding to: 


fy  pregiess 
f= f, and | f= GSE, _  (12D.25) 
P : P 
AL=0 AL >0 


The location of these frequencies with respect to the discrete spectrum 

is shown in Figure 12D-$ for 3 =e $10. This figure should be compared to 
those in Part 4 of this section. From the figure it can be seen that the 
frequency £ 4/3 has the fewest unique samples of all the sampled frequency 


components. For a, = 3: the critical Fourier frequencies are 
1/7 fo 2/7 £5 3/7 f. 


as shown in the figure. Only the first component corresponds to a sync 
frequency. 

As stated previously, the total number of independent samples for any 
given sync frequency is Ba It does not improve resolution to sample for 
an observation interval longer than, Ty =a,At- There is an exception to 
this however if the sinewave is corrupted by noise. With a signal that 
has additive noise, the resolution accuracy can be improved by increasing 
the observation period because this will provide an averaging effect for 


each of the unique sample values defined for the noiseless case. 
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3 5/14 3/8 7/18 2/5 


1/4 


Figure 12D-9. The Locations of the Principle Sync 
Frequencies as Referenced to the 


Sampling Frequency, f., 


5. Noisy data 


When “ideal" data is corrupted by noise or unwanted statistical 
variations, the collective parameters which characterize the process 
become functions of a random variable. The estimated averages will fluc- 
tuate according to how the data is taken and how much data is observed. 

The effect of noise can be reduced only by increasing the amount of 
data taken and applying averaging techniques or by smoothing the data 
record according to some a priori knowledge of the generating process. 


Quite often, data is smoothed by applying digital filtering to remove 
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the high frequency noise mixed with the desired data. In spectral analysis, 
noise effects are reduced both by selective filtering and by spectrum 
averaging. 

Selective filtering (also called notch, bandpass, highpass, lowpass, 
or rejection filtering) can be used when the desired spectrum is limited 
to some specified frequency range or ranges. This type of processing 
requires a knowledge of the "possible" spectral characteristics of the 
signal. Selective filtering and preprocessing of the data function were 
discussed in Chapter IX. 

Spectrum averaging can be implemented when several spectral estimates 
for the same process are available. In the discussion of the Blackman- 
Tukey method of spectral analysis in Chapter IX, it was noted that the 
estimated spectrum is the convolution of the "true" spectrum with the 
spectral window function (Equation 9E.5). This is only true for the 
expected value of P(f). In other words, P(f), is a random function and 
several spectral estimates must be averaged before the composite estimate 
approaches the ideal case specified by Equation (9E.5). In this sense, 
even the smoothed estimate of the power spectrum given by E{P(£)} is still 
a very idealized concept. 

There are essentially two ways to determine an average frequency 
spectrum in time series analysis. First, if the time series is used to 
generate a discrete autocovariance function which is then Fourier analyzed, 
the resulting discrete power spectral density function will be a set of 
spectral amplitudes and discrete frequencies. Several of these power 


spectral density records may be averaged to determine an average power 
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spectrum, In some instances it will be necessary to average the absolute 
values of the amplitudes to avoid the conceptual problem of dealing with 
theoretically impossible negative components. This can also be handled 
by windowing or other such techniques. 

Second, if the amplitude spectrum is calculated directly, the Fourier 
coefficients will be complex and one must chose one of two methods of 
averaging complex numbers. The first type of average is called the vector 
or coherent average. This processing involves averaging first the real 


parts and then the imaginary parts of all the complex numbers. It may be 


written as: 


(12D .26) 


c, = a, + iby, 


It is called a coherent average because it is usually used when the phase 
information is important. The second type of complex number average is 


called the algebraic or incoherent average. For this processing, one 


averages the absolute magnitudes and then the phases of all the complex 


numbers. The algebraic average is represented by 


+7 8 
= n 
Ave{c } =c_e (12D.27) 
where: 
1 N 

c == = fe | (12D.28) 

n =N k=l c;, 
ee ee (120.29) 

n N,-, kK es 
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The equations for the vector or coherent average have been worked out 


using the familiar Fourier series notation given in Equation A2.2 in 


Appendix II. 


The reader is cautioned to realize that the time records 


must be phase connected or measured with respect to the same absolute 


phase reference or the advantage of a coherent average cannot be realized, 


A particular data set denoted by the subscript, m, is corrupted with 


noise and is written as 


where the tilde accent denotes a measured value and ¢ 


noise time series for the m-th record, 


x (leat) = x(kat) 


a 


mk 


(12D.30) 


denotes an additive 


The estimated Fourier coefficients 


for the m-th record and the corresponding error contributions may be 


written 


as: 


mo 


g 


oO» 


2N+1 k=eN 


l 


+N 
a 


IN 
z 


—t 
2N+1 | 


z 
k=-N 
a 
a, + 4a 


2 


+N 
IN+1, = 


k=-N 


ry 
. + Aeon 


x (kat) 
1 +N 
x(kat) +3757 6 
2N+1 tate 


WO : an 
x (ka ) cos G nkat) 


% (kat) sin ( nkat) 


Sk 


(12D.31) 
(12D.32) 
(12D.33) 
(12p.34) 
(12D.35) 


(12D.36) 


(12D.37) 
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a 
The coefficients a, and o are the estimates obtained without noise cor- 


ruption and are denoted by: 


ee 27 akat) 

a, NGL c x(kat) cos G (12D.38) 
+N 

eae ae in (2 

b  Oye1 ee x(kat) sin G nkAt) (12D.39) 


The contributions to these estimates caused by the noise corruption are 


given by: 
+N 
ha =e OE 6, cos (7 nkat) (12D.40) 
m 2N+1,_°, uk T : 
+N 
Ab = —t x 6, sin (22 nkat) (12D.41) 
mm 2N+1 k==N mk T 7 


If there are MM independent records to be averaged, the average values 


for the estimated coefficients are calculated as: 
A ae A 
ave{a 3 = oa 7 a 


=a +ha (12D.42) 


A 
Ave {bs = 


+ Ab_ (120.43) 


The error term Aa, and Ab. can be calculated from: 


a | . 2n 

a==> > 4a == £sa ZT * ¢€.,, cos (— nkat) (12D.44) 
n MM pe Th 2S MM eal 2N+1 k=-N rk T 

b= = » = i = : ar cos (22 nkat) (120.45) 
n MM eel rT «€6MM r=l 2N+1 k=-N rk T 
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Actual data records are numerically processed by computing aon and 
bn for each record and then numerically averaging over all MM records to 
obtain ave(a__} and ave(b _}- The uncorrupted estimates of the spectral 


amplitudes are obtained by solving Equations 12D.42 and 12D.43 to get: 


A = A fa 3 us — 
a ve —_ Aa, (12D .46) 
b = ave{b_} - Ab, (12D.47) 


As the number of data points becomes very large, the estimates converge 
to the "true" values, aa and bo: The error terms Aa, and Ab. can be 
statistically specified if the statistical parameters of the noise series, 


Cok? is known. This will allow one to obtain error limits and variance 


a 
estimates on ay and 6 


A similiar analysis for the algebraic or incoherent average can be 
derived. Starting with an absolute magnitude estimate for each spectral 


component of the m-th data record we obtain: 


A fy 21,42, 02,,,4 2 2 4% 
|¢ | =p laj+bl+2ca aa +B ab) taal + Abe (12.48) 


The incoherent amplitude average becomes: 
ave([é_[}== 5 |é_| (12D.49) 
mn MM 7 


The reader should contrast this estimate with the coherent amplitude es- 


timate given by: 


1 Al, p2,5/-4—— A a 
5 [ai +b) +2(a fa +b Ab.) +ha*+ab,"] (12D.50) 
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The advantage of a coherent average over the incoherent average for ob- 
taining an amplitude resolution limit will be discussed in the chapter on 


interpretation of estimated spectra. 


6. Detection of a single sinewave in bandlimited white Gaussian noise 


A single frequency sinewave can be detected in bandlimited white Gaus- 
sian noise if the signal-to-noise ratio is large enough. In statisciesl 
detection theory (Whalen, 1971) the presence of a sinewave is determined 
by the change in the output of an envelope detector. Since the noise alone 
has a Rayleigh distribution and the signal plus noise has a Rician distri- 
bution it is possible to determine the probability of detection as a func- 
tion of signal-to-noise ratio for a given detection threshold. In spec- 
tral analysis the same types of effects are involved. A presentation of 
some basic ideas for statistical detection of periodic components in an 
estimated spectrum are presented in Chapter XIX. A simple presentation 
for a single sinewave and "ideal" noise will now be given. 

To begin, we define a time series that is the sum of a Single fre- 


quency Sinusoid and a white noise sequence: 


+j an kAt -3 SE teat 
x(kAt) = E(kAt) + Lee tee 7] (120.51) 


The white noise sequence (Appendix I, Part G) was defined as: 


E{E(kAt)}=0, E{E(kAt)E(pAt)}=0 , k#p 
(12D.52) 


2 2 
E{E(kAt) }= Oe = Py 


‘The questions to be answered are: 
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1. What are the important characteristics of the amplitude spectrum 

estimated from one or more sample records of the process? 

2. What is the effect of increasing the number of samples? 

Define a sample realization {x(kAt): k=0, +1, +2, ...,4+N} of the time 
series in (12D.51) as (2N+1) samples taken at an interval, At=1/f.. As- 
sume that no aliasing occurs and that for any sample record the average 
value is approximately zero. A discrete Fourier transform applied to the 


sample time series will give (Equation 19D.3): 


eee 


aie 
A a oe J ONS 1 
<a> Wei | = Oe 
A * 
ie Ts : ore bes oma (12.53) 
i°*t, ae 


{n=0, +1, +2, eees aN} 


Representing the noise components with Equations 19C.3 and 19C.4 and the 


-5¢, 
estimated amplitudes of cy in the complex form Be gives: 
A __1 a, ~3, 
C= oer (%,-i 8) +BLe (12D.54) 
A’ A 
= eat +B cosg j- iat + B sin %3 (12D.55) 


For a large number of samples the spectral mixing formula gives a 


good approximation for the amplitude cy: The large sample approximation 


is 
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-j¢ -je 
= 1 

B,e le le 

~jd_ +36 
Bie = ele (12.56) 
B =O n#41 
n 

and: 

A _ _1 A A 
¢ = ayer i) n#¥+1 (12D.57) 
A __l A ee: ~ 58) 
= Ser 7 (3 8) ler le (12D. 58) 


If a large number of phase connected samples are available, a coherent 


average (Equation 12D.26) may be taken which yields: 
A 


8 
* = 
Ave{e_ }= e(2}- j EtReT: 0 (12D.59) 
a, 8, 
Ave(e, }= Biggs + le, | cos 8, }- “jE + lc, | sin 8,3 
= c) (12D. 60) 


This performance is very good but it takes large amounts of data. If the 


complex amplitudes are incoherently averaged the resulting estimate 


a 


@ 8 
IR l= (gp? + Get? agi (12.61) 


is Rayleigh distributed (Equation 19C.7). The resulting estimate for 


le, | has a Rician distribution (Whalen, 1971): 


4 A 


2 8 
lc, | = Uebe + 1 le, | cos 8) + oo + le, | sin9,)7 J? (12D.62) 
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A statistical criterion for detection can be developed using the con- 
cepts of average spectral density and the Rician density function. The 
Marcum Q-Function is used to test the value of le, | against a threshold 
determined by Ave{e, }. The author plans to develop this idea in a future 
paper. 

The same concepts can be applied to Equation 12D.54 the only effect 

~i8 -i¢ 
is the change of lc, le 2 to B,e : caused by the spectral mixing effect. 

The reader is referred to Chapter XIX, Section C where more properties 

of the DFT and Gaussian noise are discussed. In particular, a relatively 


simple criterion is given for testing a noise Spectrum to see if it con- 


tains a periodic component. 
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XIII. PREDICTIVE DECONVOLUTION 


An analog or digital filter convolves an input, x(t), with the filter's 
impulse response, h(t), to give an output, y(t). The central problem of 
deconvolution is: given the output y(t) and either x(t) or h(t), can the 
third function be uniquely determined? The answer is very definitely no. 
Deconvolution involves finding the inverse filter for h(t) and this demands 
that the reciprocal of the transfer function of h(t) exist for all frequen- 
cies of interest. 

To provide a practical basis for the problem of deconvolution it is 
necessary that the class of possible filters be restricted to those for 
which the inverse filter exists and is well-behaved. This restriction may 
appear to be too severe but it turns out that much scientific data process- 


ing and spectral analysis problems can be modeled with such a class of 
filter. A class of filters variously called prediction error filters, 


digital Wiener filters, least-squares inverse filters, least-squares decon- 
volution filters, prewhitening filters, or recursive digital filters have 


been successfully used in the deconvolution of geophysical data (Peacock 
and Treitel, 1969). 

Predictive deconvolution techniques are usually applied to a degen- 
erate class of time series characterized by a known component and a purely 
random component. The desired objective is to take a measured time series, 
x(t), and decompose it into a. completely predictive component, z(t), and a 


random or uncorrelated component, 7(t). 
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In the analysis of practical time series and especially when applying 
autoregressive analysis techniques, it is assumed that the time series 
{z(t): t=0,+1,42, ...} can be characterized by a finite number of unique 
parameters (these will be discussed shortly). The uncorrelated component 
for these analysis techniques is represented by. the white noise sequence 
{E(t): t=0,1,2,...}3. Characteristics of a white noise process with 
finite variance are discussed in Chapter XIV. 

For the rest of this chapter, the time index {t=0,+1,42,...} will 
be used as shorthand notation for {t=kAt: k= 0,+1,4+2,...}. This nota- 
tion is common in the literature and so is the convention of assuming 
At=1. This convention will be used except where the explicit omission of 
At may cause confusion. 

The three most commonly used mathematical models for representing a 
measured time series are the Fourier series, the autoregressive series, 
and the convolution of the known and random components. The Fourier series 
representation may be written as the sum of a white noise sequence (see 
Equation 14.3) and a complex Fourier series with complex amplitudes, c. 


and Fourier period Tt: 


a = t 
x(t) =§(t) + 2 ce = §(t) +2z(t) (13.1) 
=-¥ 


The predictable component, z(t), is characterized by the 2M+1 complex 


amplitudes and the Fourier period, Tp This constitutes the finite number 


of unique parameters for this representation. 
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The second representation is an autoregressive series (see Chapter XIV 
and Appendix I) of order K: 


K 
x(t)=€(t)+ ED ax(t-n) (13.2) 


n=1 


The predictable component is characterized by the finite number of ampli- 
tude coefficients fa: n=1,2,...K}. The power spectral density function 
for this process model is both continuous and bandlimited. The resulting 
autocovariance function is infinite and nonperiodic. 

It is reasonable for the reader to wonder at this point if a process 
can be modeled as the sum of a Fourier series and an autoregressive series. 
The answer is, yes, a process that has a power spectral density function 
that has both continuous and discrete components should be represented as 
the sum of both models (Cooper and McGillem, 1971, Ch. 6). The autocor- 
relation function for such a model will be infinite and contain periodic 
and nonperiodic components. 

The third common model is the convolution of a white noise sequence, 
n(t), with a completely known time series, z(t), to give: 

y(t) =2(t) *n(t) = YE 2(n) n(t-n) (13.3) 
n 
It is this model that is used by Peacock and Treitel (1969) to develop a 
scheme of predictive deconvolution. The notation y(t) is used for this 
model to avoid een fusion with the autoregressive process model which yields 
similiar results. 
The desired least-squares deconvolution filter which characterizes 


z(t) is the one which ideally transforms y(t) into a white noise sequence 
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and hence 

y(t) * b(t) = z(t) * 7(t) * b(t) = n(t) (13.4) 
or: 

y(t) * b(t) = n(t) (13.5) 
The deconvolution filter and the predictable component are now related by: 

2(t) * b(t) =1 | (13.6) 


If both z(t) and b(t) are modeled using predictive filters, b(t) can be 
referred to as the inverse filter characterizing z(t). This is why the 
technique is often called least-squares inverse filtering. Peacock and 
Treitel (1969) develop a general technique whereby the data function y(t) 
is used to generate a filter b(t) which describes the deconvolved component, 
z(t). z(t) can be described as a filter or convolving function which 


transforms a white noise input, 7(t), into the time series, y(t). What 
this really means is that the spectral function representing y(t) is 


limited to a class of functions that can be described as the output of a 
real filter with white noise input. Also, the inverse filter mst exist. 


Before continuing, we will digress to examine Equation 13.1 and to put it 
into an alternate but especially interesting form. First substitute 


Equation 11.12 into (13.1) to get: 


. 277 22 
F MON tera 7 t 
x(t) = E(t) + = LS x(katje e (13.7) 
ING 1 
n=-M k=-N 


By interchanging summation this can be written as: 
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_ 277 
+N : 4M oaerer ss) 
x(t)=§(t)+ © x(kat)i=sy = e } (13.8) 
k=-N n=-M 


Now the kAt dependence is factored out of the term in brackets and the 


series is written in a more compact form as 


+N 
x(t) =€(t)+ F m (t)x (kat) t=0, tat, +2At, ...4NAt. (13.9) 
k=-N 
where: 
a ~§ 27 n(kat - t) 

a(t)=ser re F (13.10) 

n=-M 
ee pe cos(22 n(kat - t) 3} (13.11) 

N+ 1 a Te ; 


it is interesting to note that (13.9) is similar in form to (13.2) except 
that the coefficients, a(t), are functionally dependent on the index ¢t. 
Also, the coefficients are even in both t and kAt and are another form of 
the estimated Kronecker delta function (Equation 11.19). This difference 
between (13.2) and (13.9) accounts for the difference in the two types of 
time series representations. The indices for the representation of (13.9) 
do not have to be symmetrical about t=0 and could just as well be written 


as 


x(t) = §(t) + : ~% (t) x (kAt) t=0, At, ... 2NAt (13.12) 
k=0 
with a(t) now being symmetrical about the point, t= (N+1)At. 
If the time series contains a single frequency spectral component 
where the period is fairly well-known, the complex Fourier amplitude can 


be obtained by using the estimator (Equation 11.15): 
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: . att 
; +N -j T kAt 
ez) “NST, E x(kat)e (13.13) 


This component is then removed from the data by (See Appendix II) 


20 


z(t) = x(t) - 2|e¢£,) [cos (FF t+9) (13.14) 
te] 


where $, is the phase angle of écz,). The numerical algorithm for this 
estimation will take the form: 


2N+1 


re(C(£.) }= go> z x(kAt) cos (= (k-N-1)At] (13.15) 
k= ° 
_y tl a . 
1 {6¢£,) }= Sea = *(kat) sin (EF (k-N- 1) at] (13.16) 
k=1 ° 
lecz,) |= [Re?{¢}+ 1m2(¢}7? (13.17) 
%, = arcrant ile (13.18) 
re{¢} 
x(kAt) = x(kAt) - 2 Iece,) | cos (s (k-N-1)At+90] (13.19) 
°o 


PROGRAM 06 in Appendix III has been written to implement the estimator 
described above. The modified time series, x(t), may also be spectrum : 
analyzed to determine a "residue" spectrum. If the period on is not 
exactly known, the complex amplitude can still be found by using approx- 
imation techniques. For example a discrete poudier transform estimation 
using x(t) would give approximate spectral components to use as starting 


values. 


We now proceed to derive the mathematical properties and methods 
involved in determining a suitable predictive deconvolution technique. 
The predictive filtering viewpoint and the autoregression analysis view- 
_ point will both be presented since they are mathematically equivalent. 


The discrete convolution formula for a digital filter (Equation 5C.4) 


t 
y(t)= UDh(t-n)x(m) t=0,+4+1,42,... (13.20) 
n 


where the lower range of the index, n, depends on the length, K-1, of the 
digital filter. The sampled fusiiee response of the filter is limited in 
time to the series {h(nAt): n=0,1, 2,...(K-1)} where h(nAt) =0 for 

k2K. K is called the length of the filter and the lower limit on n 
becomes (t-K+1). The filter coefficients have been normalized by At. 
Linear prediction can be formulated in terms of a filter output by con- 
sidering h(t) to be a linear prediction operator with prediction distance 
a@At. The eneeie of the filter will be an estimate of the input time series 


at some future time t+a. This is represented by: 
t A 
y(t)= 2 h(t-n) x (n)=x(tt+a) (13.21) 


The best linear prediction in a least-squares sense is made by a filter 
with coefficients that minimize the total mean-square prediction error: 
2 A 2 
M=25 e (t+a) = Wx(t+a) -x(t+a)] t=0,+1,+2,... (13.22) 
t t 
The error of the prediction, ¢(t+a), is the difference between the "true" 


and predicted values of the time series: 


ct 
e(tt+a) =x(tta) -x(tta) =x(t+a) - 5 x(n) h (t-n) (13.23) 
n 
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The total mean-square error can be written in terms of the infinite time 


series and the filter coefficients: 


M= 2 [x(tt+a) - 
t 


n 


e 2 
x x(n) h(t-n)] 


(13.24) 


A system of K equations can be used to determine the K unknown filter co- 


efficients that will minimize M. 


the following minimizations: 


ou = 
dh(Ik) 


0 


k=0,1,2,..., (K-1) 


The k-th linear equation is derived from 


t 
SE = -2 5 [x(t +0) - 5 x(n) h (t-n) k(t -k) =0 
an(ky = ~* ¢ 
This may be written in the more usual form as 


t 
DEC Ux(n)h(t-n)Jx(t-7) = Ox(tt+o)x(t-7 


t n 


or: 


K-1 


= h(n) r(t-n) = Ux(t+a)x(t- 7) 


n=0 


If we make the following substitutions 


r(Tt-n) 


r(t+ a) 


T=0,1,2,...,K-1. 


t 


20,1, 254655 K=2 


x x(t) x (t- T+n) 
t 


2 x(t+a) x (t- 7) 
t 


t 


This system of equations is obtained by 


(13.25) 


(13.26) 


(13.27) 


(13.28) 


(13.29) 
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we can write the system of equations given in (13.28) in terms of the co- 


variance matrix and filter coefficients as (remembering r(7) = r(-T)): 


r(0) r(1) oe « ». T(K=-1) h(0) r(a) 

r¢cl) r(0) ee 6 « C(K-2) h(l) r(a+ 1) 

. = ee ee lS ‘ é 

r(K-1) r(K-2) . . r(1) r(0) | h(K-1) r(atkK-1) (13.30) 


The covariance matrix is made up of elements: 
r(t) = © x(t)x (t- 7) =xr(-7T) (13.31) 
t 


The system of equations represented by (13.30) is solved for the filter 
coefficients by inverting the covariance matrix. The least-squares pre- 
diction operator represented by h(t) can be used for the prediction: 
; t 
x(tt+a) = U h(t -n) x(n) (13.32) 
n 


The prediction error operator is defined as 
@w- 1 zeros 
——~_ 
1,0,0...,0,-h(0), -h(1),..., -h(K-1). 


{g(n) } 


(0), g(1), ..., g(a+K). (13.33) 


which may be used directly to obtain the error of an @-step prediction: 
atk 
e(t+a) = © g(n)x(tt+a-n) (13.34) 
n=0 
Equation 13.34 is also an alternate representation of (13.23). 


The least-squares prediction operator characterized in Equations 13.30 


and 13.32 contains the elements of a prediction filter while the least- 
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squares prediction error operator of (13.33) represents a prediction error 
filter. Before looking at the central problem of predictive deconvolution 
and filtering we will digress to consider the autoregression approach to 


the same analysis. 


For autoregressive processes modeled by (13.2), the best linear one- 
step predictor in a least-squares sense (Appendix I and Koopmans, 1974) 


is given by: 


K 
x(t+1)=x(t+ DH-§(t+1)= & a x(t +1-~-n) (13.35a) 
n=1 


=a, x(t) +a, x(te1l)+... +a, x(t - K) (13.35b) 


The one-step prediction error is the expected value of the mean-square 


error: 
A 2 2 _ 2 
E{((x(t+1) -x(t+1))"}=E(E (t+) }=e¢ (13.36) 


Koopmans (1974) has shown that the best a-step predictor in a least-square 


sense for an autoregressive series is (wSK+l1): 


3 a-1 A K 
x(tta)= 25 ax(tt+a-n) + x ax(tt+a-n) (13.37) 
n=1 n=@ 


R(t+a)=a, K(tto-1) +a, eMtta-2)+... a, ee41) 


+ a x(t) ta px(t> 1+... ayx(t+ a-K) (13.38) 


The a-step prediction error is (@22): 


a-1 
E{(x(tta) -x(t+a))*}=ElE(t+0) J=or[l+ £ a (13.39) 


n=1 
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The coefficients {a,: n=1,2,3,...K} of the autoregression can be ob- 
tained by solving the matrix of autocovariances specified by the Yule- 
Walker equations and derived in Chapter XIV (Equation 14.47). In the 


notation of this chapter, the solution is: 


r(0) r(1) . » L(K-1) =e r(1) 
r(1) r(0) - » r(K-2) r(2) 
r(K-1) xr(K-2) . . r(0) r(K) (13.40) 


For a cne-step predictor (a=1) Equations 13.40 and 13.30 give the 
same coefficients. The relationship between the filter coefficients and 


autoregressive coefficients is expressed as: 
h(n-1)=a_ n=1,2,3,...,K (13.41) 


Although we see that the predictive filter and autoregression are two 
seemingly separate viewpoints on prediction and prediction error for time 
series, they produce identical results for the one-step predictor. This 
equivalence will become important later on when maximm entropy spectral 
analysis is derived in terms of predictive filters and autoregression. 
The deconvolution operator, b(t) is determined using the one-step 
prediction operator and the condition given by Equation 13.6. The basis 
for the existence of this operator is the fact that the autocorrelation 
function for y(t) differs from that for z(t) only by a constant scale 
factor. This will now be shown. Using an analogy to Equation 5B.8, we 


know that the spectral density of y(t) is equal to the product of the 
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spectral densities of z(t) and 1r((t) 
S_(w =S_CwS_(w 13.42 
go) =S, CoS, Cu) (13.42) 
and consequently the convolution of their autocorrelation functions is: 
= * 
Ryy6) = RC ¥R (1) (13.43) 


From theory we know that the autocorrelation function for white noise is a 


delta function with strength oe 
R (m= ee &(7T) (13.44) 
7 i . 
The convolution integral of (13.40) becomes 
R(t) =o R(t) (13.45) 
yy? On “ez : 


and we see that the output and input autocorrelation functions differ only 

by a constant amplitude scaling factor, one We can also conclude that the 

power spectral density function amplitudes differ by the same scale factor. 
The property of (13.45) has been used by Peacock and Treitel (1969) 

to develop a predictive filter for deconvolving z(t) and nt). The matrix 

of (13.30) is augmented by adding the estimated variance of the white noise 


sequence (see Equation 14.44) 


K 
ee =r(0)+ © r(n) h(n-1) (13.46a) 
7 n=1 
K 
= R09) pe Ry 6D) h (n- 1) (13.46b) 


to give: 


r(K) 


— 
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r(l) .. rR) 


r(0) .. r(K-1) 


r(K-1) . . r(0) = 


1 


-h(0) 
-h(1) 


~h(K-1) 


(2) 


(13.47) 


The column vector [h] forms a one-step prediction error operator which 


operates on (r1 to give the one-step prediction error, one The aug- 


mented matrix equation (13.47) is identical in form to that determined by 


augmenting (14.46) or (13.40) to give: 


r(0) 
r(1) 


r(K) 


r(1) « + (RK) 
r(0) « « x(Ke-l) 


x(K-1) .. xr(0) 


q 


uA bo 


(13.48) 


Again we see that predictive filtering and autoregression are equivalent 


viewpoints. 


By comparing (13.48) and (13.47) with (13.5) and (13.6) we 


see that the deconvolution filter or predictive deconvolution operator is 


characterized by the following system of equations: 


r(O) 
r(1) 


r(K) 


r(1) « « xr(K) 
r(0) e « r(K-1) 


r(K-1) .. r(0) 


ray 


(13.49) 
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The deconvolution operator is seen to be just a scaled version of h(n) or 


a as: 
n 
1 1 
La ~ 2 by ~ 2 
oe o, 
(13.50) 
dL =-a, bo = h(n~ 1) 


Deconvolution is performed using Equation 13.5. The power spectral density 
function for y(t) is characterized by z(t) or the inverse of b(t). This 


will be shown in Chapter XIV. 
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XIV. AUTOREGRESSIVE SPECTRAL ESTIMATION 


The estimation of a continuous power spectral density function using 
autoregressive techniques is based on the fact that any weakly stationary 
(covariance stationary) stochastic process with a continuous spectrum can 
be represented by an infinite moving average (Koopmans, 1974). 


A moving average process is defined by the series 


+co 
y(t) = 5 b&(t-n) t=0,+1,42,... (14.1) 


n=-@ 
where the coefficients, b> are restricted to: 


ae 
Eb <e (14.2) 


n=-0 


The time series {y(t): t=0,+1,+2,... } is completely characterized by 
the coefficients {bo} and can be thought of as a linear transformation of 
the white noise process represented by &(t). The white noise time series 
{E(t): t=0,+1,+2, ... } characterizes the white noise process and is 
completely described by its mean, autocovariances, and spectral density. 


The mean of the white noise process is zero 
E{§(t)}=0 (14.3) 


and the autocovariances are: 


C(k) =E{(t) § (t-k)} = (14.4) 


The power spectral density function is the Fourier transform of the auto- 
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covariance function: 


oie -jwkat 2 
S-(w) = at £ C(k)e 4 =o At (14.5) 
§ k==-0 : 

The reader should be made aware that the notation in this chapter 

will differ from that in Appendix I. Also, subscripted variables and 
functions may appear with actual subscripts or with an index in parentheses. 
The sampling interval in all equations will be represented by At. 
For practical analysis considerations, the time series generated by 
a moving average process is one-sided: 
K 
y(t) = Eb e(t-n) (14.6) 
n=0 
The process and the resulting time series are characterized by the K+1 
coefficients {b.: n=0,1,2,3,...K}. The value of y(t) depends only 
? 
upon the b and K+1 present and past values of §(t). 
The time-series for a one-sided moving average process can be synthe- 
sized numerically by first generating the white noise sequence with a ran- 


dom number generator and then doing the linear operations described by 


Equation 14.6. This gives 
y(t) = bo S(t) + b, §(t -1I)+... +b, §(t - K) (14.7) 


for all t=0,+1,+2,.... 

The power spectrum for a moving average process is obtained by using 
a digital filtering analogy. A digital filter convolves the white noise 
sequence with the filter coefficients {b 3 to produce the output time 


series. Since the spectrum of the input white noise sequence is flat, the 
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spectrum of the output time series will be the amplitude response of the 
filter multiplied by the constant input spectrum. 

A look at the eeeeineian and z-transform properties of a digital 
filter will help provide better understanding of the autoregressive spec- 
tral estimator. The output, y(t), of a digital filter is the convolution 


of an input time series, x(t), with the filter weights {b, 3: 


+o 
y(t) = £ byx(t-k) t=0,+1, +2, ... (14.8) 
k==@ 


To obtain the filter weighting sequence we define the impulse response of 


a linear filter, g(t), as the filter output in response to a delta function 
input. In the sampling domain, the sampled impulse response or weighting 


sequence is: 
b = = g(nat) (14.9) 
at ® : 


n 


From z-transform theory we know that the convolution theorem in the trans- 


form domain (Equation 5A.8) is: — 
Y(Z) = G(Z)X(Z) (14.10) 


The next step is to define what is meant by the transfer function, G(Z). 
From basic filter theory we know that a complex transfer function, 


G(w), is defined for sinusoidal inputs (x(t)=sinwt) by using the relation 
y(t) = |e(w) |sin(ur+o(w)) (14.11) 


and for complex sinusoidal inputs (x(t) =e") aan as: 
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y(t) = |e(up [etIOC Fue (14.12) 


In the same manner, the transfer function for the digital filter is ob- 
tained by applying a complex sinusoidal input (x(t) = et JM) and factoring 
the output into the product of the input and a transfer function as shown 
in (14.12). When this is done for the convolution sequence given in (14.8) 


the result is: 


eo : 2.5 +0 oe 
a per iadtekae) _ .+jut E be jwkat (14.13) 


== k=-@ 
The complex transfer function for the digital filter can now be identified 


as: 


+o , 
G(w)= be JWkst (14.14) 


k==0 
The filter weights or sampled impulse response can be determined from the 


transfer function by the transform: 
+1 r 
b, = Sf ccape IPRAE ay (14.15) 


The single-sided z-transform for a causal filter is determined from (14.14) 
by substituting z=et sunt to generate the Z-transform of the filter 
weighting series: 
S on 
G(Z) =B(Z)= xX bZ (14.16) 
n=0 7 
The continuous spectral density for the moving average process can 

now be determined by driving the digital filter with white noise and 
determining the output spectrum. The relationship between input and output 


spectral densities (Equation 5B.8) becomes: 
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2 
$C) = |ecw) ["s, Cw) 
= |ccw) ’s_cw) (14.17) 


Using (14.5). and (14.14) the final result is: 
40 - 2 
2 -jwk 
S (w)=ato | © be?” (14.18) 
y k=-0 
Likewise, the power spectral density for the one-sided process becomes: 
K 2 2 
S (uw) =Ato”| Z be a (14.19) 
y n=0 7 
The continuous spectral density can also be obtained by evaluation of the 


z-transform on the unit circle: 


$(u) = ato” [B(Z) |”, z=e 3M Jul st (14.20) 


The continuous spectral density function for a moving average process is 
periodic with period 2n/At. Usually only the principal spectrum, |w| sn/At, 
is of interest. 
We now define a time series {x(t): t=0,+1, +2,... } as representing 

a weakly stationary (covariance stationary) finite autoregressive process 
if it satisfies the K-th order autoregressive series: 

K 

= a x(t = n) = §(t) t=0,+1,42... (14.21) 


The autoregression must satisfy the conditions (Koopmans, 1974) 
E{x(s)§(t)}=0 for all s<t (14.22) 


and: 
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K 
= 2 
a,=l, a, #0, aan oe (14.23) 


The finite autoregressive series is now written as: 


K 

x(t) - 2 a x(t - 2) = E(t) (14.24a) 
=1 

X(t) - a,x(t- 1) -a,x(t-2)~...-ax(t-K) = §(t) “ (14.24b) 


The power spectrum for a finite autoregressive series can be obtain 
both from a digital filtering analogy and from a one-to-one correspondence 
with a moving average process. A digital filter operating on the time - 
series {x(t): t=0, +1, 42... } produces the white noise sequence 
{E(t): t=0,+1,+2,... }. This special type of filter is referred to as a. 
prewhitening filter by most authors (see Chapter IX). It is a simple 
matter to define the relationship between input and output spectral densi- 


ties as 
2 
Su) |ACu) | =S_(a) (14.25) 


and consequently: 


2 
7 1 ae (14.26) 
ACW 


to 


1 
S_(y =-= 7 S$ 
is |ACw) \? 5 


The z-transform of the digital filter is 


K 
AZZ)= baz” (14.27) 
n=0 
and the transfer function in the frequence domain is obtained by evaluating 


A(Z) on the unit circle: 
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Jaca) | = [acz) | =| : ae Jun Be (14.28) 
g=aeti ust n=0 
Koopmans (1974, p. 218) has shown that if A(Z) has no zeros on or outside 
the unit circle, the linear filter can always be inverted and 1/A(w) is 
Square integrable. 

The equivalence of a finite moving average and a finite autoregressive 
series can be shown by applying a Fourier series expansion for the inverted 
filter and constraining the Fourier coefficients to those which will sat- 
isfy the boundary conditions of Equations 14.22 and 14.23. The infinite 


moving average formulation is 


= lecuy ferI9 (14.29) 


and it follows from Equations 14.1, 14.12, 14.13, and 14.14 that the finite 


autoregressive series can be expressed as a moving average: 


+o 
x(t) = = b,&(t-k) (14.30) 


k=-0 


In order for the moving average to satisfy the boundary conditions in 
Equations 14.22 and 14.23 it is necessary and sufficient (Xoopmans, 1974) 


that: - 
E{x(s) & (t)}=0 (14.31) 


for s=(t-1), (t-2),... This can be shown by taking the expected 


value using Equation 14.30: 
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+2 
E{x(s)§(t)} = = db. E{§(s-k) § (t)} (14.32) 


k=-o 


Using the identity, E{E(s-k) & (t)}=6(s-j- to’, (14.32) can be further 


reduced to show: 


oa 2 2 
= b, &(s-j-t)o=b._o°=0 (14.33) 


and this implies the boundary condition: 

b =b,=b ,=...=0 (14.34) 
For s=t, the expected value in (14.32) becomes 

E{x(t) § (t)}= boo” (14.35) 
but from Equation 14.24a we know that 

E{x(t) § (t)}=E{E"(t)}= 0° | (14.36) 


so that b= 1 is the final boundary condition. 
In conclusion, a weakly stationary time series satisfying Equations 
14.24 will have a one-sided moving average representation given by: 
K 
x(t) =1+ 2 b&(t-n) (14.37) 
a=1 ™ 
For a covariance stationary finite autoregression to exist, it is suffi- 
cient that all the zeros of B(Z) lie inside the unit circle. The z-trans- 
form of a one-sided moving average is related to the z-transform of the 


autoregression as: 


A(Z) = TERN (14.38) 
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The continuous power spectral density function for a finite auto- 


regression is written as: 


K 
S_(w)=orat|2 +E pe Jem ae? (14.39) 

n=1 

Or: 
7 t 
S (uw) = ——2 48 —____ (14.40) 
ll- 5 a e Jun ati? 
n=1 7 


The stationary autocovariances for the autoregressive process are defined 


as: 


C(k) =E{x(t) x (t - k) }=C(-k) (14.41) 


The Yule-Walker equations (Equation Al.31) relate the autoregression co- 
efficients, a> and the stationary autocovariances: 


K 


C({k) = & a C(k-n), k=1,2,...K (14.42a) 
n=1 
C(k) = a,C(k - 1) +a,C(k-2)+... a, C(k - K) (14.42b) 


A useful estimate of the variance of the white noise sequence can be ob« 
tained by combining Equations 14.36 and 14.37 to give: 
K K 2 
E{x(t) € (t)}=Ef{x(t) 5 ax(t-n)}= 5 a C(k) =o (14.43) 
n=0 n=0 
This estimate may be written as: 
K 


o7 =C(0)+ F a_C(n) (14.44) 
n=1 
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To obtain the estimated power spectral density of a finite sample 
fx(t): t=0,1,2,... (M-1)} of the "true" time series {x(t)}, the estimated 
autocovariances, Ck), and the estimated autoregressive coefficients, a 
mist be determined by appropriate numerical estimation procedures. 

One commonly used approach to this estimation problem is to compute 


eStimated autocovariances from the definition: 


M-k 


Ack) = ot EH x(t) x(t - k) (14.45) 
t=0- 


Next the Yule-Walker equations are used to define a linear set of equations 


A 
in &é (x) and as: 


é(1) éco) day... beR=1) 24 
é(2) Aa ao) &x-2) a, 

; = : ; : : (14.46) 
&(K) &¢K-1) é(K-1) eae &c0) a 


For initial calculations, the order of the process has been estimated as K. 
The KXK covariance matrix is inverted and the autoregressive coefficients 


are computed from: 


“1 
dco) . 2. cel) &c1) 


fay 2 2. AeK2) (2) 


&x-1)° - - 0) cK) (14.47) 


The spectrum of the estimated white noise sequence is generally not a con- 
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stant unless the sample is very large. The estimated autocovariance 


function for the approximate white noise sequence is written as: 


#(k) =E{&(t) 2 (t-k)} (14.48) 


Substituting the autoregressive series of Equation 14.24a for both 265) 


and et -k) gives 


K K 
BR (k) = Ef z a x(t-n) ae x(t-p-k)} 
n=0 p=0 P 


A 
= a ca E{x 
n 


(t-n)¥(t-p-k)} 
n=0 "p=0 P 


K K . 
=s sz ad ecn-p+k) (14.49) 
n=0 p=0 a? 


for k=0, +1, +2, coe +(M-K). 
The estimated spectral function $C) can be obtained from BC) by using 
a lag window function and a digital Fourier transform. A spectral density 


estimate for the white noise sequence can also be obtained using Equation 


14.44 to give: 


K 
ay 2 dca) (14.50) 


n=0 
The estimated spectral density for these numerical techniques can be 


written as 


¢ Cw) 
$ (a) = A (14.51). 
jl-5 ae Jen ate 


n=] 


or: 
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2 : 
$ (a) = —SAe—___ (14.52) 
ae 
Ju-s 2 eTiunaty . 
n=1 ® 
A 
Other methods of obtaining best estimates of Ack) and a, may also be used 
effectively. Most methods employ a least-squares estimate that also con- 
strains the covariance matrix so it will remain positive definite. This 


will constrain the spectral estimators to be positive over the entire 


frequency range. More about these estimators and the proper constraints 


will be given in Chapter XV. 
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XV. MAXIMUM ENTROPY SPECTRAL ESTIMATION 


A. Introduction 


A new criterion for estimating the continuous power spectral density 
function for a random time series (Stanley, 1975) was first proposed by 
Burg (1967). This estimator, called the Maximum Entropy Method (MEM), 
essentially maximizes the average entropy (Newman, 1977) over the informa- 
tion bandwidth using the estimated autocorrelation function. MEM was de- 
veloped as a technique to minimize the effects of observer bias and reduce 
certain types of processing error on the spectral estimates. For example, 
one of the most noticeable differences between MEM processing and other 
commonly used techniques is that no window function selection is required. 

It is important for the reader to realize that MEM spectral analysis 
is not meant to be a replacement for other techniques and it is not vastly 
superior to others. As with any estimation technique, MEM has its own 
strengths and weaknesses based upon an assumed process model. When reading 
the published literature on MEM it is most important to carefully assess 
the various assumptions and boundary conditions used to derive the pub- 
lished results. 

As discussed previously in Chapter IX, all spectral estimators must 
use a particular model to fit empirical data. In addition, most apply 
smoothing and all apply extrapolation or a periodic extension to the data 
to obtain infinite data functions which conform to the selected criteria. 
Once an infinite estimate is obtained, the spectral density can be found 
using basic theory (Chapter IV). A summary of the commonly used models 


will help to put MEM spectral analysis in proper perspective. 
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The model for Fourier transform spectral analysis is based upon a 
periodic extension of the empirical data function. Constraints on tnis 
extension include assuming that the observation period Ty is equal to the 
‘Fourier period Th that the signal is bandlimited, and that the number of 
discrete spectral components does not exceed the number of data samples 
(adequate sampling). This model gives excellent results for Many applica- 
tions but can have serious deficiencies if the true process is not periodic. 
In Fourier transform spectral analysis only the amplitude spectrum is 
usually computed because the power spectrum is a trivial extension (Chapter 
XII, Section C). Many users object to using a periodic extension assump- 
tion and an amplitude spectrum for analyzing data. These users often 
prefer the Blackman-Tukey method. 

The Blackman-Tukey model for spectral analysis assumes a zero exten- 
sion of the estimated autocovariance function. This zero extension assump- 
tion is emphasized by the use of a lag window function which guarantees a 
zero value beyond some chosen lag value Ta Although this assumption is 
often unrealistic, it gives good results in many applications. Users of 
the Blackman-Tukey method most often complain about lack of spectral 
resolution, inefficient computation and use of empirical data, and nega- 
tive spectral components. 

A rigorous discussion of the properties of the various spectral esti- 
mators was given in Chapters VIII, IX, and X. As a rule the most commonly 
used power spectrum estimators compute an estimate of the autocovariance 
function, apply a window function to this estimate, and obtain the power 


spectrum from the Fourier transform of the result. This technique produces 
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an estimate that has the following deficiencies: 
1. The inverse transform cannot reproduce the original estimated 
autocovariance function. 
2. The lag window introduces spectral smoothing which decreases 
Spectral resolution. 
3. The lag window aes produce a modified autocovariance function 
which could result in negative spectral components. 
4, Most estimates of the autocovariance function use less data as 
the lag value increases thereby increasing statistical variability. 
5. The estimate of the autocovariance function may be badly truncated. 
6. The arbitrary choice of a lag window function may tempt the ob- 
server to choose the one that produces the "most agreeable" 
spectrum. 
All of these deficiencies can be minimized or eliminated by using the MEM 
spectral estimator. The following results are obtained with MEM: 
1. The estimate is constrained so that the inverse transform will 
exactly reproduce the original estimated autocovariance function. 
2. MEM does not explicity use a lag window function and empirical 
test results show it produces a frequency resolution superior to 
other methods when short data records are employed. 
3. MEM constrains the autocovariance matrix to be nonnegative def- 
inite so that only positive spectral components are produced. 
4. MEM incorporates nearly all of the input data function for each 
step in the autocovariance estimate and hence reduces the problem 


of statistical variability for large lag values. 


Si 
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MEM produces an estimated autocovariance function that can be ex- 
trapolated to any desired order of lag and consequently eliminates 
the truncation problem. 

Since MEM is a data adaptive method, the selection of a lag ine 
dow is implicit in the process and its use is transparent to the 


observer. 


The maximum entropy criterion for spectral analysis will not reduce the 


intrinsic spectral estimation errors (Chapter IX) of aliasing, statistical 


variability, and lack of stationarity. These continue to be important 


practical problems associated with all spectral estimation methods and 


must be dealt with on an individual basis. 


The Maximum Entropy Method of spectral analysis has the following 


limitations and known deficiencies: 


1. 


The validity of the spectral estimate relies on the assumption 
that the measured process can be adequately modeled by a finite 
autoregressive series. This requires a "noise-like" data function 
and a spectrum that is both continuous and bandlimited. 

The observer must select an arbitrary order for the autoregressive 
process that is used to model the data. This selection intro- 
duces observer bias into the estimate because it controls the 
amount of spectral smoothing and hence resolution. A nonsub- 
jective solution to the problem of selecting an optimum order has 
not yet been solved (Parzen, 1969). 

For estimations using a large order sabovendeetina: the MEM algor- 


ithm may become numerically unstable. This occurs because the 
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locations of the zeros of the characteristic polynomial shift 
slightly due to numerical truncation and fall very near or on the 
unit circle. 
In spite of these deficiencies, the MEM technique represents a powerful 
tool for the estimation of time series spectra. Along with FET, Blackman- 
Tukey, and autoregressive techniques it should become a widely used method 
of data analysis. Its properties and applications will now be presented 


along with some derivations of fundamental concepts. 


B. Fundamental Descriptions for MEM 


The Maximum Entropy Method can be simply described as a technique for 
obtaining an infinite autocovariance function from finite data. The em- 
pirical time series is modeled with a finite order autoregressive series 
and the resulting spectral estimate is constrained by maximizing the 
average entropy. Practical algorithms may compute either the estimated 
autocovariances or the autoregressive coefficients. The power spectral 
density function is obtained by taking the Fourier transform of the auto- 
covariance function or by using the autoregressive spectral estimator. 

More than one viewpoint can be used to summarize the fundamental 
characteristics of MEM spectral analysis. The three most common view- 
points will now be discussed. The original viewpoint is that MEM is 
equivalent to maximizing the average spectral entropy of a Gaussian, band- 
limited time series subject to the constraint that the estimated power 
Spectral density and the estimated autocorrelation function form an exact 
Fourier transform pair (Chen and Stegen, 1974; Newman, 1977). This can 


also be interpreted as determining an optimum power spectral density func- 
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tion that keeps the entropy of the empirical time series stationary with 
respect to the selection of extrapolated values of the estimated autocor- 
relation function (Radoski et al-., 1975). 

A second viewpoint is that MEM is equivalent to an "optimum" method 
for extrapolating a truncated autocovariance function (Veltman et al., 1972). 
The criterion for "optimum" is that the entropy of the time series model 
must be maximized at each step of the extrapolation. Van den Bos (1971) 
has demonstrated that this criterion is equivalent to fitting the data with 
a finite autoregressive process model with a least-square-error criterion. 
This is sometimes referred to in the literature as fitting an all-pole 
model to the data. This nomenclature comes from the z-Transform descrip- 
tion of the transfer function used to model the power spectrum (Stanley, 
1975). This type of estimator contains only poles (no zeros except at the 
origin) in the z-plane. 

The third viewpoint is that MEM is equivalent to modeling the desired 
spectrum with the output of a filter driven with white noise. The empiri- 
cal data function is processed to obtain the estimated filter spectral 
response. The filter has been variously termed a prewhitening filter, a 
Weiner prediction filter, and a least-squares inverse filter (see Chapter 
XIII). An equivalent mathematical model for the empirical data function 
is the convoltuion of a finite autoregressive series and a white noise 
series (Radoski et al., 1975). In this type of processing the input time 
series is converted to a white noise sequence (See Equation 14.24a) and 


the prewhitening filter is used to characterize the spectrum (Ulrych, 1972b). 
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A fourth viewpoint, which is actually a combination of the others, is 
that the entropy loss of the filter should be maximized subject to the con- 
Straint that the inverse transform of the MEM spectral estimate yield the 
exact autocorrelation values. As with the third viewpoint, a filter is 
found that can generate the correct autocorrelation function when driven 
with white noise but it must also reduce the entropy by a maximal amount. 
This viewpoiut emphasizes the First Principle of Data Reduction (Chapter 


VII) because it minimizes the biasing effects of missing data. 
C. Entropy Definitions for MEM 


The concept of entropy is used in many scientific disciplines. Entropy 
as a thermodynamic state function in statistical mechanics is a measure of 
the randomness of the system. Entropy quantifies the lack of knowledge 
about the exact state of the system at any instant in time. In information 
theory (Beckmann, 1967) the entropy of a source of information is defined 
as the average self-information per state of the system. In commmication 
systems the entropy is often called the average information rate in bits- 
per-symbol. The entropy of a signal will be maximum when all states or 
symbols are equally likely. Two concepts that will be used in MEM deriva- 
tions are those of a entropy power and entropy loss in a linear filter. 
Mathematical definitions will now be given for the various entropy quanti- 
ties. 

The entropy of a discrete system of N states, EY with probabilities, 
Py? is given by: 

N 


H= se Pp, t2(P,) (15c.1) 
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If H is maximized subject to the constraint that the sum of all probabili- 
ties is equal to unity, the distribution of probabilities that yields max- 


imum entropy is given by (Beckmann, 1967): 
1 
B= 5 n=1,2,3,...,N (15C.2) 


A convenient definition for the entropy of a continuous distribution 


of a single variable is (Goldman, 1953; Beckmann, 1967): 
H=~-f p(x)4n(p(x))dx (15¢.3) 


If H is maximized subject to the constraint that the variance of x remains 
constant, the calculus of variations solution gives a Gaussian distribution. 


The entropy of a Gaussian distribution is given by: 
H= ta(2neo")* (15.4) 


It is also true that, for a constant entropy, a Gaussian distribution has 
the smallest variance of all possible distributions. 

An excellent discussion on the effect of a linear transformation on 
entropy has been given by Goldman (1953). In essence Goldman shows that 
the entropy of a signal vector Ys produced by a linear transformation of 


the vector x5 is given by 
H =H _+4ala,.| (15¢.5) 
y x tJ 
where la; ,| is the determinant of the transformation matrix defined by: 


N 
vo= Bap at (15C.6) 
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Random noise may be characterized by sampling points in the time domain 


which are statistically independent and have distributions which are Gaus~ 


sian. The entropy per degree-of-freedom for random noise is 
h_=tn(21e 02)? (15¢.7) 
n n 


where o is the variance of the noise. To obtain the total entropy it is 
necessary to multiply by the Nyquist frequency, fy = W/2at, and the dura- 


tion or observation period, T,, of the signal: 
HL 2f Ty ho (15C .8) 


The entropy power (Goldman, 1953) of a signal is: 
1 an 
P =s—e (15¢.9) 
For Gaussian noise the entropy power is equal to the variance. Entropy 
power is always less than or equal to the time average power. 


Entropy loss in a linear filter is a function only of the frequency 


response of the filter and is most easily described in the frequency demain. 


The derivation for entropy change in a linear filter is analogous to that 
for a linear transformation since filtering is a linear transformation 
process. For N sampling points in the frequency domain the entropy at the 
output, Hy» is related to the entropy at the input, HL. using (Goldman, 
1953) 

y 2 2 

H =H + = 4a(c- lect.) |% (15C.10) 
y x ° n 

n=1 

where G, is the filter gain and G(f) is the normalized frequency response. 


In the limit as the frequency sampling interval goes to zero and the 
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frequency response becomes continuous, the entropy change becomes: 
H =H +1, f tn(G" |e) |2) ae (15¢.11) 
y x N ‘ fe) a 


The entropy~per-degree-of-freedom can be obtained by dividing (15C.11) by 
2ety? 
fy 
ho=h+ sf tn(olecty |? ae (15¢.12) 
y x 2f, fo) 


The change in entropy is obtained both from the filter shape and the filter 


gain. The integral can be separated into: 


2 
ram in|c(£)| a£+ta(c,) (15¢.13) 
Oo 


In most of the literature and for the rest of this text the filter gain is 
assumed to be normalized to unity and only the integral term is considered. 


The entropy power at the output of the filter is given by: 
P =>~e 7 (15¢.14) 


If the class of power spectra to be analyzed by MEM is restricted to 


those that can be represented by a linear filter of finite length, we can 


define an average spectral entwopy density or spectral entropy rate as: 
+ 


h tals, (£) Jaf (15¢.15) 


1 
x" te J 
Wag 


The power spectral density function, S,.(£) is two-sided and analogous to 


Iece) |7. This is the definition that will be used for MEM (Lacoss, 1971). 
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D. Mathematical Bases for MEM 


A good mathematical description of maximum entropy spectral analysis 
can be presented using any or all of the viewpoints discussed in Section B. 


Three somewhat independent derivations will now be given. 


1. Maximum average spectral entropy basis 


The basis first proposed by Burg (unpublished 1967) maximizes the spec- 
tral entropy subject to the constraint that the inverse Fourier transform 
gives the exact values of the finite autocorrelation function. For a Gaus- 
Sian process with a continuous bandlimited power spectrum, this entropy 


measure (Equation 15C.15; Smylie et al.,.1973) is represented by the inte- 


gral: 


b= Tay I 7 tals (£) lat (15D.1) 
Restrictions on S.(£) are that it be nonzero in the Nyquist frequency in- 
terval and that it be representable as the power transfer function of a 
linear filter. S,(£) is considered to be the "true" power spectral density 
of the process and ty is the maximum frequency of nonzero spectral con- 
tent. This would be the folding frequency for a sampling system or the 
Nyquist frequency for a bandlimited signal. 

From previous theoretical considerations we know that the power spec- 
tral density function and the autocorrelation function form a Fourier 
transform pair as represented by the Wiener-Khinchine relations (Equation 


44.9). From transform theory it follows that the inverse Fourier transform 
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of the continuous and bandlimited spectral density produces an autocorrela- 
tion fenebeon which may exist over the entire autocorrelation domain. It 
also follows that the autocorrelation function cannot be finite in extent 
because two finite fener tons cannot form an exact Fourier transform pair 
(uncertainty principle). If either the autocorrelation function or the 
spectral density are exactly known, the other may be computed using the 


transformation: 


thy 
"F “Ms (4) j= f s_ (fe IT ae =R(1) (15D.2) 
“fy 


A capital R is now used to represent autocorrelation but it has the same 
meaning as small r in Chapter XIII. 

Of course for practical problems the data function is finite and only 
an estimate of the true power spectrum can be obtained. A finite data 
function produces an estimated autocorrelation that is both truncated and 
Statistically variable. Both of these important effects must be accounted 
for in a practical MEM spectral estimator. As a start we will first con- 
Sider the problem of truncation. 

To be able to satisfy Equation 15D.2 the truncated autocorrelation 
function must be extrapolated in some optimum manner. To examine this 
possibility the autocorrelation function is separated into a "known" com- 
ponent, R(T), defined in the interval In| st, and an "unknown" component 
R (1) representing the autocorrelation for all lags In| >. The inverse 


transform relationship of (15D.2) can now be separated into the two com- 


ponents: 
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ee 
+j2ntt 

Je 8 (He df =R (7) In| <7, (15D.3) 

<2. 

i § (fe Tae = R (1) In] >r, (15D.4) 


Tee will be defined as the maximm lag at which values for R(T) are 
known. The so-called "known" component plays a dual role in MEM analysis. 
For developing the theory it is assumed to be a truncated version of the 
exact or "true" autocorrelation function while in actual application it is 
an estimate based on empirical data. For the analysis to follow it will 
be assumed to be exact while in Sections F and G it will be an estimate. 

.When the maximum entropy estimate satisfying (15D.3) is obtained, the 
"unknown" component given by (15D.4) is completely specified. It can be 
referred to as a maximum entropy extrapolation of the autocorrelation func- 
tion. From this viewpoint the maximum entropy criterion is equivalent to 
the a priori assumption that the "best" spectral estimate must be the most 
random of all possible estimates that is still consistent with the "known" 
(but truncated) autocorrelation function. This also means that the extra- 
polated part of the autocorrelation function adds the least amount of "in- 
formation" to the spectrum. 

The derivation that now follows is based on the calculus of variations 
material presented in Part J of Appendix I. The entropy measure that will 
be maximized is given in Equation 15D.1 and the constraint equation will be 
a sampled version of (15D.3). The extra degrees-of-freedom which allow hy 


to be maximized are represented by Equation 15D.4. 
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First, the use of a sampied version of Equation 15D.3 must be justified. 

Now because we are interested only in S(f) defined in the interval {£| sf, 
it is possible to represent S(f) either by Equation 15D.2 or as a periodic 
function with period 2fh- The mathematical restriction imposed by this 
representation is that we never try to use it for [£| oi The fact that 
a continuous and periodic spectrum can be represented by an infinite sum 
of sampled autocorrelation values (Stanley, 1975) leads to the selection 
of the following infinite series and basis set for S(£): 

s_(£) ae R,(karyeJ2TKOT (gc 1) (15D.5) 

k=-0 

The sampling seeiéa in the autocorrelation domain is related to the spectral 
width by At= 1/2f.. To verify that this basis is valid, it is substituted 
into the following sampled version of (15D.2): 

thy 


+j2rinat 
Je S (fe 


df= R(adT) (15D. 6) 


After substitution, integration and summation can be interchanged to give 


the form: 


aes + 
ard a ceat) f[ |eti@nt(n-k) Ar 


k= -@ 


di= R(na7) (15D.7) 


To show that this is an identity, the form of the integral is compared with 
the representation of the Kronecker delta function given in Equation A2.33. 
This representation is obtained by identifying f= 1/247 and T= 2f5 to give: 


+ 
ee a SPT (15D.8) 


“Ay 
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The identity in Equation 15D.7 becomes 
+o 
ot Reb) 6 = RL (nat) (15D.9) 
and we have shown the representation in (15D.5) to be valid. 
A corresponding sampled representation for Equation 15D.3 is obtained 
by defining a maximum known lag index M= 7 /At. This gives 2M+1 known 


lags to define the bandlimited spectrum. The constraint equation and cor- 


responding series definition for &(£) become: 


Ge, 
i § (eet? minb tag = R (mbt) (15D.10) 
{n= 0, +1, +42, ..., +4} 
+M 2 2 
$ ceyear ER Cearye J27EKAT (15D.11) 
x kam * 


From the calculus-of-variations solution and Equations 15D.1 and 


15D.10 we can identify the following terms for the Euler-Lagrange equation: 


F(£,8 (£)) = qe tal (£)] (15D.12) 


“fy 


= +j2nfinAt 
Q,(£,8,(£)) = 8 cee (15D. 13) 


The Euler-Lagrange equation is: 


+M : 
rs) A 
Crc#,8 (£))- ZA» (mq _(£,8.(£))]=0 (15D. 14) 
ad (£) x n=-M x n x 


x 
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The partial derivatives are 


= _ —_ 2 = etj2mindt (15D.15) 
#58 (9 


and the equation is evaluated to be: 


M : 
; ay oS A (n)et Jenends =0 (15D.16) 
4e§ (£) nem ; 


From this solution we see that both the Lagrange multipliers and the inverse 
spectrum must be finite. This reaffirms the constraint of nonzero $(£). on 
MEM spectral estimates. 

The next step in the solution is to solve: the Euler-Lagrange equation 
for $ (£) in terms of the Lagrangian multipliers, A(), and frequency, f. 
This solution is substituted into the constraint equation and the 2M+1 
values of the autocorrelation function are used to solve for the 2M+1 
values of A, (2) 5 the Lagrange multipliers. The solution is completed when 
the A (2) are used in the explicit equation for § (£). The explicit eqaua- 


tion is obtained by rearranging (15D.16): 


+M ‘ -1 
§ Cf) = zt z a (net I2rEnAr (15D.17) 


n=-M 
it is pointed out at this time that Equation 15D.16 is a finite and sampled 
version of Equation 15E.6. 
The direct method of solving 15D.17 is to use a Laurent series expan- 
sion and equate it term-for-termm to Equation 15D.11. An alternate method 


which derives the spectral estimate directly in terms of the autocorrela- 


232 


tion coefficients used in the Burg algorithm (Section G) will be presented. 
The derivation essentially follows one given by Ulrych in Smylie et al., 
(1973). 


“The z-transforms of Equations 15D.17 and 15D.5 are equated to give: 


ATR(z) = zae J2TEAT (15D.18) 


The polynomial, A(z), represents the double-sided z-transform of the series 
in Equation 15D.17 and, as such, is an ideal (not estimated) representation 
of the Lagrange multipliers. Conceptually, R(z) is also the infinitely 

long Laurent series expansion of 1/A(z). If the z-transform of (15D.11) is 
represented by atR(z), then R(z) and Rez) mist have equal coefficients for 
like powers of z. We also know that 4, (a) = 0" (-n) and R (nat) =R. (-nA7) 

because § (£) must be a real function. Since these coefficients are Hermi- 
tian, the polynomials R(z) and A(z) can be factored into the product of two 


polynomials and Equation 15D.18 can now be written as (Smylie et al, 1973): 


1 


ArC(z)C (1/z2") = — (15D.19) 
G£.Q(2)Q"(L/2") 
The new polynomials are defined by: 
* * 
R(z) =C(z)c (1/z ) (15D.20) 
Mz) = Q(2)Q"(1/2") (15D.21) 


The symbol Q for the polynomial must not be confused with that for the 


generalized function in the Euler-Lagrange equation of (15D.13). A similar 
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process for obtaining a polynomial product is the spectral factorization 
for rational spectral densities discussed in standard texts on signal anal- 
yais (Cooper and McGillem, 1971). 

The system of equations which represent the maximum entropy solution, 
for example Equation 13.48 in matrix form, can be written in series form as 


K A 
Re a, (aR, (k +n) = ~P,6 () (15D.22) 


where a, (0) =-1 and 65 (%) is the Kronecker delta function. The mean-square 
error of the estimate will be represented by Pe which is equivalent to the 
tati o 
fe) re) 
notation oe 
We now implement a mathematical technique that will help simplify the 
solution. The system of equations represented by (15D.22) is augmented with 


the arbitrary series 


h(k) =0 [05 TF OKT 
(15D.23) 
#0 {k=-1, -2, -3,..., -K} 
to give: 
K A 
= (n)R (k+n) =-P,6 (k) - h(k) 
ag ee ee (15D.24) 
{k=0,+1, +2, ..., +K} 
The z-transform of (15D.24) is the start of the solution: 
A(2)R(z) =P, + H(z) (15D.25) 
2 K 
A(z)=1-a,(1)z-a,(2)z - +207 a, (K)z (15D.26) 


H(z) =h(-1)z7}+h(-2)277 4... +h(-K)27* (15D.27) 
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A(z) is the z-transform of the digital filter (Chapter XIV) characterizing 


the autoregressive process. 


A 
Since R(z) also consists of Hermitian coefficients it likewise can 


be factorized into the special form 
A * * K 
R(z) =U(z)U (1/2 ) = U(z)V(z)/z (15D.28) 


where the product polynomials are defined by: 


U(z) =u tuyzt eee +u,z* (15D.29) 
* * Kel * 
V(z)euce tue. +... buy (15D. 30) 


Substituting (15D.28) into (15D.25) and dividing gives the desired result: - 


K 


PLZ N , 
A(2)0(2) = geey + BOS (15D.31) 


The rather elaborate means by which we have arrived at Equation 15D.31 are 
necessary so that the concluding steps, including a polynomial division, 


are easier. 


Typical terms on the left-hand side of (15D.31) are: 
A(z)UCz) = ao# fu, - a,(1)u, e+ OE fu, - Ua ® 2% sate 
- [u,a,(K -1)+ a, (Ku, ast - Ua (K) ae (15D.32) 


This is a polynomial of degree +2K. Typical terms for the polynomial 


divisions on the right hand side are: 


p2* K P h = 
(z) K -1 ° Z : 
es S75 = fC = ,)2 eR ea (15D.33) 


u 
© 6 US 
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Since the exponents on the left-hand side are positive and those on 
the right-hand are negative, equating equal powers of z on both sides of 


Equation 15D.31 gives the trivial resuit: 


u = (15D. 34) 
bat) 


The coefficients of all other powers of z are zero so (15D.31) reduces to 


the result: 


Px 
A(z)U(z) a ae (15D. 35) 
er 
In a similar manner we can also obtain the result: 
* k o* * * PK 
A (1/z )U (1/z )=u, Sa. (15D. 36) 
fo) 


Combining these results and relating them to (15D.28) gives the z-transform 
A 
of RCkat) in terms of the z-transform of the autoregressive series: 


P 
R(z) = u(z)U"(1/2") = ——&—__ (15D.37) 
A(z)A (1/z ) 


Substituting this result into Equation 15D.11 and evaluating the z- 


transform on the unit circle between 0 and m gives the maximum entropy 
v 


estimate of the power spectral density function: 


ATP, 
== =a (i5D.38) 
i J2mEA7) 10, j2ntat. 


§ Cf) = 


A( 


From Equation 15D.26 we get the identity 
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~j27£kAt (1sD.3 


ie) 
ad 


e J2mEAT 


K 
AC JHl- 2 (k)e 
k=1 *K 


and Equation 15D.38 can be put in the usual form given by Equation 15F.1. 


In the cause of academic curiosity we can determine the Lagrange mltipliers 


from 
MZ) = = A(Z)A (1/2") (15D.40) 
K 
where: 
K -n 
MZ) = = (nz (15D.41) 
n=-K 


The solution is obtained by equating like powers of Z in the equations: 


E Ae elie £ ky § ay 
(n)Z “=L1- (k)Z 1l- £ (k)Z (15D.42) 
n=-K * k=1 *K k=1 *K 


This completes the derivation of the maximum entropy spectral estimate. 


Alternate derivations are given in-the next two parts of this section. 


2. Autocorrelation function extrapolation 


The following description of the MEM spectral estimator is based on 
the desirable property that the autocovariance matrix for a real finite 
autoregressive process must be semipositive definite (Veltman et al., 1972). 
This implies that the determinant of the autocovariance matrix must be non- 
negative. This condition guarantees that the resulting estimated power 
spectral density function will be nonnegative over the interval lf| <4. 

The analysis using the autocovariance matrix is based on the fact that it 


can be used to obtain the estimated spectrum by applying the Fourier trans- 
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form directly or by generating the autoregressive coefficients (Chapter XIV) 
and using the autoregressive spectral estimator. 

As in the previous section we will assume that the first 2N+1 auto- 
correlations are known exactly and that the process is Stationary and 
Gaussian. The maximum entropy extrapolation problem is to find the (N+1) 
autocovariance value by maximizing the entropy defined by the autocovari- 
ance matrix. 

To simplify the matrix notation for this section we will use the ab- 
breviation r(n) = R, (nar). The autocovariance matrix of order N will be 


represented by R(N) where the matrix elements are defined by: 


Rj =r(j-i) (15D.43) 


The autocovariance matrix for an (N+1)-th order autoregressive process 


is given by: 


r(O) r(1) . . © r(N) 
r(-1) r(0) ... r(N-1) 

R(N) = |... .2..-26-e. (15D.44) 
r(-N) r(-N+1) . r(0) 


For real processes the correlation function is even, r(n) = r(-n), so it 
would be possible to rewrite the matrix with all positive subscripts. To 
emphasize the fact that the function is double-sided we will keep the nega- 
tive notation for the moment. 

The autocovariance matrix, R(N), is extrapolated to one order higher, 
R(N+1), by maximizing the entropy measure defined by (van den Bos, 1971 


and Smylie et al., 1973): 
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h= tn{(27e)s* aeefRcn +1) 7 (15D.45) 


This is the entropy of a N+2 dimensional Gaussian probability density 
function with covariance matrix R(N+1). To maximize h with respect to the 


unknown correlation, r(N+1), the first derivative is set to zero: 


dh _ & dadet [R(N+1)] _ ; 
ee = (15D.46) 
ar(N+ 1) (one) get CRON + 1] ar(N +1) 


This is equivalent to finding r(N+1) such that: 


adet [(RN+D] _ 4 (15D.47) 


ar(N+1) 


From Equation 15D.46 we see that R(N+1) must be positive definite so that 
det[R(N+1)] is positive and nonzero. The derivative of det (R(N+1)] is 
obtained by first reducing R(N+1) to the next lower order (this is the 
matrix minor of element Ry 42 ,N+2> and then applying a lowering operator to 
all indices. This gives: 


r(-l) xr(O) ... xr(N-l1) 
r(-2) r(-1) . . . 0 0r(N-2) 


sR ET de Varia, Big cece Re aero, Lesa 9G ASD ES) 


ar(N+ 1) 
r(-N-1) r(-N) ... rl) 


Two additional useful properties of det{R(N+1)] can also be demonstrated. 


N 


The first is that it is a quadratic function of r(N+1): 
det [R(N+1)]=0,+0,r(N+1)+ ant (N+ 1) (15D.49) 


This is obtained by actually expanding det{R(N+1)], setting r(-m) =r(m), 


and collecting terms. The coefficients Oo» As and a, are functions only 


a) 
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of the elements of R(N). The second property is that the second derivative 


of det(R(N+1)] is always negative 


2 
odet[RN+1)] __, det [R(N - 1)] (15D. 50) 


ar-(N +1) 
because det(R(N-1)]>0. From the quadratic representation we can immedi- 


ately identify the coefficient a, to be: 

a, = -det[R(N-1)] (15D. 51) 
The quadratic now takes the forn: 

det{RIN+ 1) J=a, +0, r(N+ 1) - fa, [e*Qv+ 1) (15D.52) 


A typical plot of det (R(N+1)] as a function of r(N+1) is a parabola that 
is concave down. A valid solution can exist only in the region det({R(N+ 1) ] 
> 0. The parabolic form also guarantees that there is only one maximm in 
the valid region. Since the ordinate values of the parabola are symmetrical 
about the maximum, it aiso follows that maximizing the entropy is equivalent 
to choosing a value for r(N+1) that lies at the midpoint (Veltman et al., 
1972) of the region where det[R(N+1)]>0. 

The final solution for the unknown autocorrelation value is obtained 
by differentiating Equation 15D.52 and solving for r(N+1): 


Sdet [RN+1)] _ a, - 2a, [rav+1) =0 (15D. 53) 


ar(N + 1) 


et %y 
r(N+1) = 2a) = 2det [RIN-1)] (15D. 54) 
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A satisfactory solution requires that aT be nonzero otherwise the extra- 
polation would terminate. The determination of oo requires several matrix 
manipulations starting with Equation 15D.48. To eliminate notation com- 


plications caused by negative indices the even property of r(n) is invoked 


to give: 


r(1) r(0) ... r(N-1) 
r(2) r(1) . .. r(N-2) 

G60? |) %. ews Sie Soe Se ee ao Se 8 (15D.55) 
r(N) r(N-1) . . r(0) 


r(N4+1) r(N) ... rl) 


Expanding the determinant using the last row and the method of minors 


produces the result: 


a 
r(N+1) det [R(N-1)] + x(k) (-1) hy KT? (15D.56) 


The Mod , are the corresponding matrix minors of the elements in the bot- 
> 


tom row. Finally we can identify the coefficient a, as: 


N 
a= r(k) I te (15D.57) 


In conclusion, the MEM extrapolation algorithm is simply a step-by- 
step computation of higher order autocorrelations to achieve an arbitrarily 
high order matrix. The matrix manipulations indicated by Equation 15D.57 
are not implemented in practical algorithms because of the computational 
effort involved. In the next section an equivalent estimate based on an 


autoregressive model is derived that has better computational efficiency. 
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3. Autoregressive or predictive-filter basis 


The third mathematical description of MEM involves the definition and 
calculation of a prewhitening or predictive filter. In Chapter XIII it was 
shown that predictive filtering and autoregression are equivalent viewpoints. 
Before starting a detailed analysis it will be helpful to briefly review 
some basic system analysis concepts that will help tie together the equiva- 
lent viewpoints. 

A continuous-time, linear, time-invariant system with a single input 
x(t) anda single output y(t) is illustrated in Figure 15D.1. The differ- 


ential equation which describes the system is given by: 


M M-1 
saa ¢ +...¢+aye= 
. - eee y 7 
ay ac! au 1 art 1 fo) 
K K-1 
dx dx 
by mi +b at +...+b x (15D.58) 


Linear 
System 


Figure 15D-1. Continuous-time, Linear, 
Time-invariant System. 


An equivalent discrete-time system, is likewise described by a linear 


difference equation (Stanley, 1975) with constant coefficients as given by: 
aya) + aya -i1) + ajy(n -2)+...+ ayy(n -M)= 


b x(n) + b x(n -1l+...+ b, x(n - k) (15D.59) 
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The z-transform definition for the transfer function of a discrete-time 


system is obtained from (15D.59) by applying a unit impulse: 


af = és 
bo +42 +b,2 +e tb Z 


-l1 -2 -M 
Z tay Foor tay’e 


K 


H(Z) = (15D. 60) 


aj? ay 
A special form of this transfer function is used in MEM spectral analysis 
and is called the "all-pole" model. The terminology "all-pole" model (van 
den Bos, 1971) in the literature refers to a evauetee function that has no 
zeros in the z-plane. As a consequence, the numerator polynomial in Equa- 
tion 15D.60 has all zero coefficients except for bo: 
if the input to the filter is white noise we say the output time series, 
y(t), was generated by a moving average process (See Chapter XIV), Equations 
14.6 and 14.19). The output time series power spectral density is completely 
characterized by the coefficients, be 
If the input to the filter generates an output which is white noise 
we refer to the input time series as representing a finite autoregressive 
- process of order M. The power spectral density function is completely 
specified by the coefficients, a. (Equation 14.40). The "all-pole" model 
is represented by the finite autoregressive series given in Equation 14.24. 
The Yule-Walker equations are used to relate the finite autoregression 
coefficients, ay? and the stationary autocovariances. These relationships 
are presented in detail in Equation 14.42. This linear set of equations 
is expressed in matrix form in Equation 14.46. After manipulating the ma- 


A 
trix form, changing the notation so C(n)=r(n), and adding a row of order 


N+1, the Yule-Walker equations can be expressed as: 
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r(1) r(0) ... r(N-l) 1 0 
r(2) r(l1) . . - r(N-2) “a, 0) 
es “f= | ° C15D.61) 
r(N+1) r(N) ... rl) “ay 0 


This is a homogeneous system of N+1 linear equations and from Cramers 


rule the characteristic determinant of the system must be zero; therefore: 


r(1) r(0) . .. r(N-1) 
r(2) r(1) ... r(N-2) 
det . oe ee we ww we ww = 0 (15D. 62) 


r(N4+1) r(N) ... ri) 


Since this solution is identical to that obtained by maximizing the entropy 
(Equation 15D.55) the autoregressive model is equivalent when the autocorre- 
lations are known exactly. If it is desirable to obtain the extrapolated 
autocorrelation values using an autoregressive model, they are easily ob- 
tained from the following modification of Equation 14.42a: 

N 

r(N+1) = 2% a xr(N+1-n) (15D. 63) 

n=1 7 
This requires a knowledge of exactly N values of a, and N values of r(k). 
The power spectrum obtained from the autoregressive coefficients is equal 
to that obtained by a Fourier transform of the autocorrelations. The auto- 
regressive technique is superior to the extrapolation method in the last 
section because the coefficients, a. are easily obtained from the known 
autocorrelations by using Equation 14.47. The practical implementation of 


an MEM spectral estimator is presented in Sections F and CG. 
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E. The Companion Function for RC) 


At the beginning of the author's research, one of the first problems 
attempted was to relate the maximum entropy viewpoint of spectral analysis 
to the Wiener-Khinchine relations (Equations 44.8 and 4A.9). The concepts 
of MEM were not well-understood and it was hoped that this approach would 
provide the needed insight. The result of this effort was the development 
of the generalized calculus-of-variations formulation for integral con- 
straints given in Part J of Appendix I. The maximm entropy measure dis- 
Seis in the previous section and its integral constraint equation are an 
application of the calculus-of-variations solution that was derived. 

This approach to the investigation of the properties of MEM led to the 
discavery of a companion function, A(T)» for the autocorrelation function, 
RL (T)> which ee a eertein class of processes gives an equivalent descrip- 
tion of the power spectrum of the process. The results that follow will 
seem peculiar because the constraint equation completely specifies S_(£) 
and thus allows no degrees of Pecans The reasons for presenting this un- 
usual approach are to show that MEM is consistent with the Wiener-Khinechine 
relations for specific classes of spectral functions and to show the dis- 
covery of the companion function. 

The entropy measure to be maximized is given in Equation 15D.1 and the 


integral constraint equation is given by the Fourier transform relationship: 
+o : 
fo sceye™?t* Tae eR (1) (15E.1) 
eey- =) 


Comparing Equations 15D.1 and 15E.1 with Equations Al.72 and Al.73 in Part J 


of Appendix I we can identify: 
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F(£,S,(£)) = tals (£)] (15E.2) 
Q(T, £,8,(£)) = 8 (£)etd27F7 (15.3) 


The factor (4£,)7" will be neglected in this section. 


The appropriate Euler-Lagrange equation is (Equation Al.83): 
+co : 
EE oe 2 ar = 4 
= J 1A 0 (15E.4) 


Evaluating the partial derivatives in (15E.4) and solving for S(£) in terms 


of A(T) and £ gives the solution: 


= __l Q_ ti2ntt 

35 * 5 (£) 3S 7 e (15E.5) 
DS a 235 +j2rtt._ 

TWH J A (te dt = 0 (15E. 6) 


Th: 
conjugate of the Fourier transform of the continuous Lagrangian multiplier, 


A(7)> is equal to the reciprocal of the power spectral density function 


for the process: 


1 _ofe 
mS =F fA(1) } (15E.7) 


This representation is restricted to processes which have a well-behaved 
reciprocal spectral density. The spectral density. mist be continuous and 


nonzero for all f. This limits the representation to stochastic processes 


when the frequency is not bandlimited. For bandlimited processes the dis- 
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cussion in Section D applies and the reciprocal spectral density is always 
well behaved. 

Continuing with the calculus-of-variations method, the solution for 
s_(£) is substituted into the constraint equation to give the relationship 


between A(T) and R(1) = 


+o E 
f = ef J2T Tae = R Cr) (15E.8) 
=o PA (1) } ; 


This integral equation can be solved for A,.(7) as a function of R¢1) by 
taking the direct Fourier transform of both sides. This procedure gives 


the following results: 


1=Fia (on PRA (1) (15E.9) 


~J2m£7 a6 (15E.10) 


+0 
1 
A_(1) = e 
x9 * J FR 
Finally the solution for A(T) is used in (15E.6) to derive equations for 
S,(£) as a function of A(t) and R(t): 
S(£) = we (1) 3 (15E.11) 
Fino 
The power spectral density function, S42), is specified completely by the 
Fourier transform of the autocorrelation function (Wiener-Khinchine rela- 
tions) or alternately by the reciprocal of the complex conjugate of the 
Fourier transform of the continuous Lagrangian multiplier function, A, (7) - 
Because of this analogy, the function ACD will be called a companion 


function for the autocorrelation function, R(T). 


247 


After a considerable amount of effort was Beets ea trying to deter- 
mine an interpretation for the companion function, a simple solution was 
seen that now will appear trivially obvious. Consider Equation 15E.9 for 
the case of a "real" process. The autocorrelation function is real and even 


and so is the companion function. The equation can then be rewritten as 


1 =Fils (7) IFAC) 3 (15E.12) 


or 


l= § (fA C2) (15E.13) 


where A (£) is the spectral function represented by the Fourier transform 
-of A(t). Comparing Equation 15E.12 with Equation 5B.8 we can make the 
following interpretations: 

1. Equation 15E.12 represents a linear filter operating on a stochastic 
input to produce a white noise output. In short: an ideal. pre- 
whitening filter. 

2. S,.(£) and R(t) represent the filter input. 

3. A (£) and A (£) represent the filter transfer fretion where 
A c£) = [ace |?. 

4. The output spectral density function represents white noise with 
unity amplitude, 5 Ao) ete 

The Fourier transform relationships in (15E.12) can be replaced by the fol- 
lowing convolution integral: 


+0 
(r)=f  RCe)A (r= t)de (15E.14) 


this is yet another way of representing the ideal prewhitening filter. 
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Equation 15E.14 is analogous to (14.21) for a finite autoregressive process. 
Also compare (15E.13) and (14.25). 

In sunmarv we can say that the companion function, AD, represents 
an ideal linear prewhitening filter and that the filter transfer function 


is specified by: 
lace) |* ="Pirce)} (15E.15) 


F. MEM for Discrete and Finite Data 


For empirical data records the data function is finite and the esti- 
mated autocorrelation function is also discrete and finite. Furthermore 
the autocorrelation function is also statistically variable with the vari- 
ability increasing with lag. The exact autocorrelations used in the pre- 
vious sections will now have to be replaced with estimates. The second 
factor that enters the picture is the selection of the order of the process 
used to do the modeling. The order affects the resolution of the estimator 
and the numerical stability of the computer algorithm. 

Practical methods of implementing the MEM spectral estimator use the 
autoregressive model and compute the coefficients directly. This approach 
has the simplest numerical implementation because it can be done recursively 
and avoids the matrix manipulation needed in other approaches. This direct 
approach is the only practical one for large orders. The presentation to 
follow will show the methods employed to control statistical variability 
and to select a suitable order. 

To begin we will define all of the terms used in the autoregressive 


spectral estimator given by (Equation 14.52) 


where: 


AT 
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8. (£) ia K 
1-<= a (ke 
k=1 *K 


pene? (15F.1) 


frequency at which the spectral density is estimated. f is 
limited to the range, \e| si: 

the Nyquist frequency or folding frequency determined by the 
sampling rate, ft 1/2AT. It will be tacitly assumed that the 
data function has been suitably preprocessed to eliminate aliasing. 
sampling interval or lag interval between samples in she autocor- 
relation domain. The lag interval is also equal to the sampling 
interval in the time domain, AT= At. 

the order of the autoregression used to model the process. For 
most applications K is never larger than approximately one-third 
of the total number of data function samples. 

fa, (Kk): k=1,2, ...,K} the estimated autoregressive coefficients 
used to model the empirical data function. A boundary condition 
for physical realizability is: a, ) <1. 

Estimated power spectral density function at frequency f for a 
K-th order sutorentessiog approximation to the process. 

total mean-square error for a K-th order autoregressive model of 
the process. This is also the one-step prediction error given in 
Equation 13.36. It is also the estimated variance of the white 


noise sequence produced by removing the predictability in the data 
; 


250 


function (Equations 14.43, 14.44, and 14.48). This is sometimes 
referred to as the total error power of the prediction. 
The system of equations used to represent the maximum entropy solution 
was discussed in Chapters XIII and XIV (Equations 13.48 and 14.47). This 
matrix formulation is repeated below in the notation that will be used in 


this section: 


70) f(1) oe. ECR) 1 oe 
r(1) (0). . E(ReL) -a,(1)| }o 
2 " : : ~ a (2))=/0 (15F.2) 
t(K)  4£(K-1). . 2(0) -a,(K)| {0 


L 
We can now get down to the crux of the problem encountered in the 
practical application of MEM, autoregressive, or any other spectral esti- 


mation technique: 


How does one obtain "good" estimates of the autocorrela- 
tions, Ron), or the autoregressive coefficients, a, (k), 
directly from the finite data function? 


To begin, consider estimating the autocorrelation function: 
r(n) =E{x(t)x(ten)} | (15F.3) 


For a finite data function defined by {x(t): t=0,1, 2, ...,N-1} an un- 


“ 


biased estimator is: 


N-|n|[-1 
A 
r(n) = Ea 2 x(q)x(q+ [n|) (15F.4) 
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It is unbiased because for large N it converges to the theoretical values 
defined by Equation 15F.3. This estimator becomes unsatisfactor for large 
lags because the amount of the data record used is so smali that the vari- 
ability of the estimate becomes intolerably large (Jenkins & Watts, 1968). 
A common solution to this that appears in much of the literature is to bias 


the estimator with a triangular window function (Schwartz and Shaw, 1975): 


t'(n)=(1 - dal tay (15F.5) 


This choice is adequate for some applications of spectral analysis and is a 
special case of the Blackman-Tukey method discussed previously. It is un- 
desirable in many applications because it causes smoothing in the estimated 
spectrum. The estimator in Equation 15F.4 may actually produce an FFT with 
negative components. In fact there is no assurance that the covariance 
matrix will be positive definite (Jenkins & Watts, 1968). 

We now proceed to demonstrate the computation of the autoregressive 
coefficients directly. An earlier paper by Levinson (1947) gives an excel- 
lent discussion on linear prediction and the least-squares prediction error 
‘criterion for obtaining the "best" filter coefficients. J. P. Burg (1968) 
modified the Levinson algorithm to give an estimate that would guarantee 
that la, | would not exceed unity (Ulrych, 1972b). A brief presentation 
of linear filtering and prediction patterned after Levinson's treatment 
will now be given followed by a derivation of Burg's recursion method. 

Linear filtering and prediction can be briefly exptained with the help 
of Figure 15F-1. As much as possible, notation will be patterned after 


Chapters XIII and XIV. A known signal x(k) is corrupted by white noise &(k) 
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to generate a corrupted signal y(k). This corrupted signal represents the 
data function in the spectral estimation process. The data function is 
then "optimally" filtered by a convolution with h(n). The "best" estimate 
of x(k) represented by (Kk) appears at the output of the filter. The esti- 
mation error for the k-th sample is represented by E(k). The following 


equations apply 


y(k) = x(k) + &(k) (15F.6) 

E(k) = x(k) - X(k) (15F.7) 

é K 

x(k) = EZ h(n)y(k-n) (15F.8) 
n=0 


where K is the order of the estimate. 


x(k) 


x(k) 


E(k) 


Figure 15F-1. Linear Filtering and Prediction 


A measure of the quality of the estimate is the expected vaiue of the mean- 
square error (Equation 13.36): 
2 x a2 
E{e"(k)}=E{[x(k)- © b(n)y(k-n)}°} (15F.9) 


n=0 
{k=0,+1, +2, ..., +} 
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Denoting this error by M, and expanding the expected value using Equation 
15F.6 gives: 
K 


M, = E{x"(k)}-2 £ h(m)E(x(k)x(k- n) } 
n=0 


K 
- 22 h(n)E{x(k)§(k - n) } 


n=0 
K K 
+2 ££ h(n)h(m)Efly(k -n)y(k-m)} (15F.10) 
n=0 m0 


By recognizing the appropriate autocorrelations and assuming x(k) and €(k) 
are totally uncorrelated we get: 
K 


M, = R,(0) - eat h(n)R, (n) 


K kK 
+ & & h(@)h(m)R (m-n) (15F.11) 
n=0 m=0 ss 
It is important to note that Ry(m- n) represents the autocorrelation func- 
tion of the generated or measured data. 
The best estimation in a least-square error sense for the filter coef- 


ficients is obtained when M, is minimized with respect to the h(n). The 


partial derivatives 


e = = 
ACE) 0 {k O51 2 feu (15F.12) 


are evaluated to give 


K 


a 2 
h(k) = “2 2 ae h(n)R, (k - n) = 0 (15F.13) 
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which for a system of K+1 equations: 

K 
2 h(n)RL(m~- 9) = R,(m) m=0,1,...,K CLSF.14) 
n=0 


The minimm error of the estimation is obtained by substitution of (15F.14) 


into the double summation of (15F.11) to give: 
K 

M, (min) = R, (0) sae h(n)R (a) (15F.15) 
The eatin filter gives a best estimate of the x(k) in a least-squares 
sense. The solution to the system of equations given by (15F.14) is pre- 
sented as an iterative technique in Levinson's paper and is referred to as 
the Levinson algorithm, This completes the presentation on linear filter- 
ing and prediction except to state that the infinite data model used would 
have to be modified for finite data before it could be applied to a prac- 
tical filter synthesis problem. 

The Burg recursion also determines the linear prediction-error filter 
coefficients and minimizes the mean-square error of the estimate. The ap- 
propriate system of equations for the Burg recursion given by Equation 
15F.2 is analogous to Equation 15F.14 augmented by Equation 15F.15. It is 
interesting to note that Equations 15F.2 can also be written in series form 
(See Equation 15D.22) as: 

E A A A2 
Za (n)r(k+n) = r(k) - a6 (Ck) [k= 05152) cas Kh (15F.16) 
n=1 
The best linear one-step predictor in a least-square error sense is given 


by (Equation 13.35): 
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: K 
x(t)= 2 a, (n)x(t -n) (15F.17) 
n=1 , 


The one-step prediction error for a K-th order autoregressive model is the 


total mean-square error: 


oe = E{{x(rt) - X(t) 4 (15F.18) 

2 7 2 

= E{{x(t)-= a,(n)x(t -n)] } (15F.19) 
n=1 


For a finite data function defined by {x(n): n=0,1,2,..., (N-1)} 
Burg suggests defining both a forward prediction error and a backward pre- 


diction error: 


K 
Ee(q) = x(q) - © a (n)x(q =n) (15F.20) 
n=1 
K 
E(q-1) = x(q-1) - a a, (n)x(q +n) (15F.21) 
n= 


To eliminate end effects the predictor subscripts must be limited to 


forward: Ksqs(N-1) (15F.22) 
backward: .Lsqs(N-K-1) (15F .23) 
inclusive: Ksqs(N-K-1) (15F.24) 


where K is the order of the estimate and N>K is the total number of data 
samples. The mean-square error of a K-th order prediction would now be de- 
fined as the mean of the error for both forward and backward predictions. 


For N data samples: 
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1 (N-K-1) , : 
M = TD ok [e-¢a) +E(q- 1)} (15F.25) 


The solution proceeds by minimizing M, with respect to the a, (n) in a re- 
cursive manner and solving the system of equations for the a(n). A pro- 
cedure for implementing the Burg recursion in a manner similar to the Lev- 
inson algorithm has been outlined by Anderson (1974). A derivation of this 
technique and a computer algorithm for implementation will be given in 


Section G. 


G. Burg MEM Recursion Algorithm 


This algorithm determines estimates of the autoregressive coefficients 
a, (n) which are "best" in the following senses: 

1. The coefficients generate a maximum entropy spectral estimate when 
used in Equation 15F.1. 

2. The entropy loss in a linear predictive filter is maximized. 

3. The estimated mean-square error of a K-th order prediction, oes 
is minimized. 

4, Equations 15F.2 are satisfied in a least-square error sense. 

5. la, (K) | camnot exceed unity (Ulrych, 1972b). 

6. The autocovariance matrix is positive definite. 

7. No zero extension of the data is needed and no window function is 
used explicitly, 


For a finite data function defined by 


{x(t): t=1,2,3,...,N} (15G.1) 
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the average mean-square total prediction error for a K-th order process can 


be written as: 


1 2 c 2 
ao x(t) - : a, (k)x(t +k) 
My.  2QN-K) = 
l Ne-K 9 
* 20K) 5 = E tx(e+e)- E a, (k) x(t +K-k) J (156.2) 


The reader should convince himself that the equation above is identical in 
every respect to Equation 15F.25. The form and subscript notation now used 
are more convenient and closely conform to those used by Anderson (1974). 
The implementation of Equation 15G.2 by the minimization of M, is done 
with a recursion which starts with order K=1 and progresses to some maxi- 
mum value K=2,3,..., Ky This procedure generates a 2x2, 3x3, ..., 
K XK, autocovariance matrix which is both positive definite and maximizes 
the entropy. By generalizing this procedure it can be shown that the auto- 


regressive coefficients are related by (Smylie et al., 1973): 


ay (k) = a, (k) - a (KR)a,_ | (K-&) (156.3) 


K=2,3, ...,Q@-)D 


k=1,2,..., (K-1) 


Using the notation, P= oe a recursion for Pe is obtained by using Equation 


15G.3 in Equations 15F.2 or 15F.16. It is: 


2 

P= Py_yll- a (k)] (156.4) 
N 

P= g = x = £0) | (156.5) 
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At each step in the recursive process Pe is the minimm value of M. at 
that step. As can be shown using (15G.3) the only degree of freedom at each 
new step is the coefficient a, (K) . To obtain the least-squares solution, 
the average prediction error should be minimized with respect to a, (K) This 


gives the following system of equations: 


se: 0 ‘Ret Syacsn. (156. 6) 
It will be seen shortly that the solution for a, (KR) which minimizes M is a 
function only of x(t) and can be calculated directly. 

Another important feature of the technique is that the estimated auto- 
correlation functions can also be obtained recursively (analogous to the 
Yule-Walker equations) as 

m 


(m) =. 3 a fa) F(a 4) (15G.7) 
q=1 


where (0) is given by Equation 15G.5. This is analogous to the system of 
equations represented by (14.42). 

A convenient formulation of the solutions represented by (15G.6) has 
been given by Andersen (1974) and will now be summarized. First define 
. a, (0) = -1 so M in Equation 15G.2 can be rewritten as: 
| N-K 


K 
Synoe ae a 2 
M = To Ae C¢ i a, (k)x(t+k)) 


K : 
ees ay(k)x(t+K = k))? (156.8) 
k=0 


Next define a, (k) =0, k2K in the recursion of Equation 15G.3 (except for 
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the term shown explicitly as a, (K)) and substitute into (15G.8). With a 


change of sign inside the squared terms the results are easily determined 


to be 
1 N-K ; 2 
M = Tq-K) Rela sae ~ a, (K)B(t)) 
+ (Bit) = a, (K)B,(t))7J (156.9) 
where: 
K 
B(t)= 2 a,_j(k)x(t+k) (15G. 10) 
k=0 
K 
B(t)= 5 a,j (K-k)x(t+K-k) (156.11) 
k=0 


t=1, 2; eos N-K 


The value of a, (RK) which minimizes M, as defined by Equation 15G.6 is easily 
determined since B,(t) and BYé(t) are both independent of a, (RK). Taking the 
partial derivatives of Equation 15G.9 as described by Equation 15G.6, we 


obtain the desired result: 


N-K 
2 = Be(t)BY(t) 

a,(K) = WK 5 3 
Bo(t)+B’“(t) 

t-1 * . 


(156.12) 


A simple example of a first order autoregression will illustrate the dif- 
ference between estimating ay (K) with the Burg technique and using the 


estimated autocorrelations. The Burg technique gives: 
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N-1 
22 x(t)x(t+1) 
_ tel 
2,1) = 5 3 ‘ 
= Ex"(t) +xft4+1)] 
t=1 


The direct method gives: 

N-1 

xX x(t)x(t+1) 
- EQ) _ _N t=1 

a, (1) A N-l N 

x= x(t) 

t=1 


(156.13) 


(156.14) 


Both estimates are unbiased. The condition la, (K) | <1 can be shown by ap- 


plying triangle inequalities to (15G.12). 


The final task that remains is to demonstrate useful recursion formulas 


for B, (t) and Be(t). Using (156.3) in (15G.10) and (15G.11) and paying 


careful attention to the valid range of indices and previous definitions 


for a, (k) outside the normal ranges gives: 
Be(t) =By y(t) a, 1 (K~ 1)BY 4 (t) 
Bye(t) = BY _j(t+1) - aj (K- DB (t+1) 


K=2, 3, cooa KD 


t=1, 2, ..., (N-K) 


= U = \ 
B,(t) x B,(t) x(t + 1> 


(156.15) 


(156.16) 


(156.17) 


(15¢.18) 


(156.19) 


This completes the presentation on the Burg MEM Recursion Algorithm except 


for a few summary comments. 
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It should be strongly emphasized that the Burg recursion reduces but 
does not eliminate the effects of statistical variability in the estimated 
coefficients. Also the effect of numerical truncation in a computer imple- 
mentation of the technique may introduce zeros in the numerator of Equation 
15F.1. The statistical variability of Ms and consequently a,(K), increases 
with K. This can be interpreted as a reduction in the amount of data used 
for the estimate. The reader can verify this by examining Equations 15¢.9, 
15G.10, and 15G.11. The quality of the estimate of S,(£) as a function of 
K is best evaluated using both P(or M) and the variance of the estimate 
of Pee We know from previous work that EP, }= og. These factors will be 


discussed more in Chapter XIX. 
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XVI. ONE-BIT AUTOCORRELATION SPECTRUM 


Although the one-bit autocorrelation scheme of digital spectral analysis 
is geo knowm and widely used in radio astronomy (Weinreb, 1963), the method 
is not presented in most textbooks. This computationally simple digital 
technique makes use of the fact that only phase information is necessary for 
the determination of the shape of the power spectral density function of a 
Gaussian process. Amplitude information is necessary only for the deter- 
mination of the amplitude scale factor of the spectral density. 

To implement this technique, the data function, x(t) is processed by 
first sending it through a zero-crossing detector and then clipping the out- 


put to obtain digital logic levels. This process is described mathematical- 


ly as 


sgn{x(t) } (16.1) 


y(t) 
or: 


+1 x(t) 20 
| (16.2) 


y(t) 
-1 x(t) <0 


The digital signal, y(t), is then sampled at a uniform clock rate to code 
the samples for digital processing. The resulting digitized time series 


can be used to obtain the estimated autocorrelation function of y(t) by the 


following correlation process: 


: N 
B_(7) =i fs y(kAt)y(kAt + T) (16.3) 
N ped 


The estimate of the normalized autocorrelation function of the "unclipped" 


signal is now obtained by the Van Vleck relation: 
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A : T A Pa 
p(t) = sin{> p(T) J (16.4) 


Finally, a normalized estimate of the power spectral density of the process 


is obtained by the Fourier transform: 
A 
§ (a) = RMB (7D) (16.5) 


The amplitude scale factor, K, may be found by measuring the power in the 
data function, x(t). 

A complete discussion of one-bit autocorrelation spectral analysis is 
given in Weinreb (1963). He derives the Van Vleck relation and also the 
mean and variance for the one-bit autocorrelation function estimate. He 
shows that, in terms of a large number of samples, N, the mean of the esti- 


mate is approximated by 
am re Gees 16.6) 
ay PS SN Py : (16. 


and the variance by: 


2 


a2 Ty ne bge 
O. ~ iy (b> ey (1- ey) (16.7) 


The hardware implementation of a simple one-bit crosscorrelator is 
relatively simple. The block diagram of a crosscorrelator designed by the 
author for radio astronomy research is shown in Figure 16-1. It is also 
possible to use the same system for autocorrelation if y(t) is a time-delayed 
version of x(t). 

For autocorrelation, it is simpler to provide the necessary time delay 
(or lag) by sampling the input signal and using digital delay techniques. 


The block diagram for a simple one~bit autocorrelator is shown in Figure 16-2. - 
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One-bit autocorrelation techniques can be applied only when large 
amounts of data are available. The method offers simplicity in exchange 
for large data sets. Smail data sets require the extrapolation and 


smoothing techniques inherent in methods such as MEM. 
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XVII. DATA REDUCTION SCHEMES 


The various computer programs written for data reduction are described 
in this chapter. Each discussion includes an introduction, a list of pro- 
gram variables, and an example run. Programs 01, 02, 03, and 04 are in- 
tended primarily to be tutorial aids. Programs 05, 06, and 07 are user 
oriented and designed to be used to process data. For this reason, the last 
programs will be explained in much more detail. Listings of the programs 
are given in Appendix III while further examples of their use are given in 
various chapters. The printed outputs for the programs are also given in 
Appendix III and the reader mist refer to them for the examples. Each pro- 


gram number is keyed to an example in the appendix. 


A. PROGRAM 01 - Discrete Fourier Transform Testing 


1. Introduction 


This program was written to illustrate the digital Fourier transform 
and its inverse as derived in the text. The program consists of two sub- 
routines which can be used to illustrate various sampling effects on the 
estimates of the Fourier coefficients. The program is provided for user 
experimentation and as an example. It is not intended for routine data 
processing. 

SUBROUTINE FSRS computes an output time series corresponding to input 
Fourier coefficients. It is a program implementation of Equation i2D.1. 
The program equations follow the more familiar Fourier series form given 


by Equation A2.2 in Appendix II. 
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SUBROUTINE LNSPIM computes the Fourier coefficients from an input time 


series. It is a program implementation of Equation 12D.5. The program 


equations are the digital form of Equations A2.4 and A2.5 in Appendix II. 


2. Input/output data definitions 


A(I) = 


B(I) 


CMAG(I) = 
CMAGDB(1) 


CPHASE (I) 


DCV = 
FX(I) = 


FVALUE (I) 


= 


a 


3. Example 


Fourier coefficients corresponding to the cosine inte- 
gral or even part of the input time series. 

Fourier coefficients corresponding to the sine integral 
or odd part of the input time series. 

Absolute magnitude of the i-th Fourier coefficient. 
Absolute magnitude in dB. 

Phase angle of the i-th Fourier coefficient referenced 
to the center of the input time series. 

Average or 'dc" value of the input time series. 

Output time series generated by SUBROUTINE FSRS. 

Input time series for SUBROUTINE LNSPTM. 

Total number of input samples of the time series (must 
be odd). 

Equivalent sampling interval in seconds. 

Fourier period, seconds. 

Observation period, seconds. 


Sample times for the output time series. 


The input for the example program is the Fourier coefficients a, =4, 


3 


b,=1. The Fourier frequency is one Hertz and 21 samples are taken in an 


observation period of 3.1 seconds. 
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The program generates its own analysis data by using SUBROUTINE FSRS 
and the Fourier period ("true" period), T,- 

The complex Fourier amplitudes are computed from these 21 samples 
using SUBROUTINE INSPIM. The user must remember that this routine requires 
neither a sampling time nor an observation period. These are only used to 
interprete the output results. 

The last step is the computation of the estimated time series with a 
factor of four increase in time resolution. This output can be compared to 
the "true" series to judge the quality of the estimate. 

The numerical results for this example are compared in Figure 12D-3. 

The quality and numerical stability of the estimate can often be 
judged by applying Parseval's theorem to the results. First, the complex 


amplitudes are determined using (A2-13): 


Second, from Equation 12.A8, the sum of the squared amplitudes gives: 
+3 , 
R(O) = = lc |? 8.50 volts~ 
n=-3 


From the estimated time series, the result approximated by (123.29) is 


1 2N+1 5 5 
A 
P= NAT eed x, 8.70 volts 


and finally, from the estimated complex coefficients (with c, = 70.132): 


M 
Bp = Is |P +2 b> el? = 8.67 volts” 


n=1 
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The estimates give good results when compared to the true value thus in- 
dicating good quality and numerical stability. Refer to PROGRAM 05 for 


more details on this method of verifying results. 


B. PROGRAM 02 - Estimated Kronecker Delta Function 


1. Introduction 

This program evaluates the approximation to the Kronecker delta func- 
tion. It is included to show the high degree of accuracy in approximating 
Equation 12D.11. Figure 11-1 shows another application of the program. The 
practical usefulness of the routine is limited to analysis and testing of 


Signal processing schemes. 


2. Input/output data definitions 


DELTQR = Matrix elements of the estimated Kronecker delta. 
M= Maximum row order. 
N = Maximum colum order. 


Q = Row index of DELTQR. 


R= Column index of DELTQR. 

RF = Fourier period in seconds. 

TN = Observation period in seconds. 
3. Example 


The example run illustrates the result when the Fourier period and the 
observation period are equal. DELTQR is defined for both positive and 
negative indices. Diagonal elements are unity and off-diagonal elements 


are better than thirteen orders of magnitude smaller. 
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Cc. PROGRAM 03 - Estimated Spectral Amplitudes using Kronecker Delta 


1. Introduction 

This program is a simple example of the application of the estimated 
Kronecker delta function to computing spectral estimates. The algorithm is 
specialized to real (zero phase angle) amplitudes and must be rewritten for 
complex amplitudes if the user so desires. The routine is intended as an 
example of spectral mixing errors caused by an observation period unequal 
to a multiple of the Fourier period. Equation 11.20 is a prototype for the 


algorithm. 


2. Input/output data definitions 


C(Q) = Estimated spectral amplitudes. 
CTRUE(I) = Input ('true') spectral amplitudes. 


M = Maximum index an input spectral amplitudes. 


N = Index corresponding to 2N+1 samples. 
Q = Index for double-sided spectral amplitudes. 
TN = Observation period in seconds. 
TF = Fourier ,period in seconds. 
3. Example 


The data function has a Fourier frequency of 1 Hertz (T,=1) and has 
been sampled 21 times in a period of 3.1 seconds. The true spectral com- 
ponents are 2.0 and 0.38 at 1 and 3 Hertz respectively. The estimated 
spectral components at zero frequency and intervals of Gay Hertz are 
listed in the printout. 


At 0.968 and 2.90 Hz., the estimates are 2.038 and 0.356 respectively. 


Although these are reasonably accurate, spectral mixing has introduced 
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several large components at other frequencies. The component at 0.645 Hz. 
is -0.253 which is approaching the size of the true component at 3 Hz. This 
effect is present in every spectral estimator and must be considered when 
evaluating the quality of the estimate. Notice also that the average value 


of the estimate is not zero. 


D. PROGRAM 04 - Discrete Fourier Transform Analysis 


1. Introduction 


This program evaluates the discrete Fourier transform in its most fun- 
damental form. The complex coefficients evaluated are those necessary for 
the complex series representation (Equations A2.8 and A2.9). The algorithms 
evaluate Equations 12D.5 and 12D.6 and also do linear regression on the in- 
put data to remove an undesired linear trend. For purposes of better evalu- 
ation, the estimated complex amplitudes are used to generate an estimated 
time function with four times more time resolution than the input. 

SUBROUTINE LNSPTM computes the complex Fourier coefficients based on 
the input time series and observation period. The algorithm is an imple- 
mentation of Equation 12D.5. 

SUBROUTINE FSRS is used to generate a time series from Fourier coeffi- 


cients and time period information. The algorithm is an implementation of 


Equation 12D.6. 


2. Input/eutput data definitions 


A(T) In-phase or "real" component of the DFT output. 


B(I) Quadrature or "imaginary" component of the DFT output. 


BZERO = Intercept point from linear regression. 


CMAG = 


CPHASE 


CMAGN(I) = 


CPHASN(I) = 


DCV = 
FREQ = 


FSAVE(I) = 


FVALUE(I) = 


FX(I) = 


LINREG = 
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Slope from linear regression. 

Component power in dB relative to unity total power. 
Magnitude of each spectral component in dB relative to 
the computed amplitude after scaling. 

Magnitude of the estimated Fourier coefficient. 

Phase of the estimated Fourier component in degrees. 
Normalized magnitude relative to unity total power. 
Phase in degrees. 

Average or de-value of the input time series. 

Frequency corresponding to each amplitude estimate. 
Initial input time series data saved for comparison 
purposes. 

Input time series data after linear regression. 
Estimated time series with increased resolution. 

A user selected data preprocessing option. LINREG = 0 
chooses the input data function for spectral estimation 
while LINREG=1 selects the time series after linear 
regression modification. 

Number of data samples for use in SUBROUTINE LNSPTM. 
Number of data samples in the input time series. 

User selected scale actos for computing spectral ampli-. 
tudes in relative dB. 

Data sampling interval is seconds. 
Fourier period in seconds. 


Observation period in seconds. 
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YBAR = Average value of the input time series. 
YF(I) = Sample values of the straight line determined by linear 
regression. 
3. Example 


Time series data for the example was provided by Mr. Wai Ling Tsui 
(Tsui, 1976). The time series is 41 samples of the output of a phase- 
switched interferometer. The radio source is Hercules A (3C348) and the 
data was taken 29 July 1975. 

The scientific objective is to obtain an estimate of the natural fringe 
frequency of the source. This is obtained by computing the digital Fourier 
transform of the time series data shown in Figure 17D-1. The resulting 
estimated spectrum is given in Figure 17D-2. The plot is smoother than the 
printed data because zero-filling (concatenation) was used to improve the 
output resolution. The peak used to estimate the fringe frequency occurs 
at 2.11x107° Hertz. The true fringe frequency is predicted to be 1.95 x 
107? Hertz. This result is typical of what might be expected for DFT spec- 


tral estimation. 
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Figure 17D-2. DFI Spectral Estimation 
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E. PROGRAM 05 - Fast Fourier Transform Spectral Analysis 


1. Introduction 

This program implements a Fast Fourier Transform (FFT) routine to do 
spectral analysis. It is intended to be used for routine data processing 
and has features which include such preprocessing options as average value 
removal and data conversion to standard variable form. Output data options 
are also included that provide suppression cf unwanted output during routine 
operations. 

The program will accept an arbitrary number of input data points and a 
resolution option provides an increase in output resolution (but not accur- 
acy) that is helpful in analyzing and plotting results. These features are 
accomplished by adding zeros (zero filling or concatenation) to the time 
series to bring the total number of input data values to a factor of. 2” 
where n is an integer. The FFT subroutine mist have 2” data points to oper- 
ate properly. Another feature is that, in addition to giving the usual 
aliased FFT output for o points, the program does proper scaling and conm- 
putes the true amplitude spectral density function. It also monitors the 
Nyquist frequency to guarantee that no output above the folding frequency 
is given. Finally, an estimate of the double-sided power spectral density 
function is given using the squared-amplitude definition. 

SUBROUTINE FFAST is adapted from a FFT routine given by Stearns (1975, 
Appendix B). The method is described as time decomposition with input bit 
reversal. 

A flow diagram describing the essential features of the program is given 


in Figures 17E-la and 17E-I1b. 
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‘USER' CHECK ; READ INPUT 
DIMENSIONS DATA FUNCTION 


PRINT 
INPUT DATA 
STATISTICS Soak eciniasietienes 


PRINT 
INPUT DATA AND 
STANDARD VARIABLES 


' PRINT oi 


STATISTICS 


é» : : S 
YES 

X(I) = XSV(I) X{I) = MMC(T) X(I) = X(I) 

FILL FFT ARRAY & 

WITH DATA 


COMPUTE 
CALL COMPUTE ESTIMATED 
ere ic aaah 
FFAST * PHASE 
COMPUTE ESTIMATED 
-— —_— — — — — 
SONEINUE POWER SPECTRUM 


Figure 17E-la. PROGRAM 05: Fast Fourier 
Transform Amplitude Spectral 
Density 


279 


Sa SS a CONTINUE 


YES YES YES 


PRINT 
AMPLITUDE 
SPECTRUM 


PRINT 
POWER SPECTRAL 
DENSITY 


‘USER’ SUPPLIED 
PLOT ROUTINES 


LOOP PROGRAM 
FOR 
ADDITONAL RUN 


SUBROUTINE FFAST 


DETERMINE 
POWER - OF - TwO 
ORDER 


FILL ARRAY WITH 
ZEROS AS 
NECESSARY 


‘USER" CHECK 
DIMENSIONS 


COMPUTE FFT 
USING TIME DECOMPOSITION 
WITH INPUT BIT REVERSAL 


Figure 17E-1b. PROGRAM 05 


2. 
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Input data definitions 


NDC = 


NRESL = 


PTOPT = 


The processing option for the input data function. The user 
has a choice of letting the FFI operate directly on the in- 
put data function X(T), (NDC=0), the zero-mean data function 
XMDC(T), (NDC=1), or the data function converted to standard 
variable form XSV(T), (NDC=2). 

The resolution option for the FFT display. A-donzere sel- 
ection will add a power-of-two resolution increase for the 
FFT output. For example, NRESL=3 will increase the output 
display resolution by a factor of 27, 

The total number of time samples in the input data function. 
The user must dimension program variables accordingly. 

A printing option used to control the printed output of the 
program and used to suppress unwanted output. PTOPT is a 
binary word of the form: (NPT1, NPT2, NPT3, NPT4). NPT1_ 
controls the printing of the input data functions. NPT2 con- 
trols the printing of the FFT complex output. NPT3 controls 
the printing of the amplitude spectral density. NPT4 con- 
trols the printing of the power spectral density. Use a 1 
if printing is desired and a 0 to suppress printing. 

A name or label in FORMAT (5A4) used to identify the data 
record. 

Sampling interval for the input time series, seconds. 

Input time series data. Units of volts will be assumed for 


simplified presentation. 
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3. Qutput statistics 


DCV = Average value or estimated mean of the data function, Volts. 

XBAR2 = Mean-square value of the input data function, Volts’. 

XVAR = Unbiased estimate of the variance of the input data finetion, 
volts’. 

STDEV = Standard deviation, Volts. 


4. Output data definitions 
ACN) = In-phase or "real" component of the FFT output, Volts. 


B(N) = Quadrature or "imaginary" component of the FET output, Volts. 
CMAG(I) = Absolute magnitude of the discrete spectral component ob- 

tained from the discrete Fourier transform (DFT), Volts. 
FMAG(N) = Absolute magnitude of the FFT output, Volts. 


F.PHASE(N)=Phase angle of the FFT output MOD mT, Degrees. 


FR(I) = Fourier frequencies of the spectral estimates, Hertz. 

KMAX = Total number of independent and nonredundant spectral com- 
ponents. 

N= 2 * KMAX, the total number of data pairs used in the FFT sub- 
routine. 

S(I) = The estimate of the double-sided power spectral density func- 


tion, Volts*/Hz (Watts/Hz in 1 ohm). The printout gives only 
the positive frequency values. 
XX(I) = The estimate of the double-sided amplitude spectral density 


function, Volts/Hz. The printout gives only positive fre- 


quency values. 
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XMDC(I) = The input time series with the average (or 'dc') value re- 
moved. 

XSV(T) = The input time series converted to standard variables which 
are derived from X(T) by subtracting the average value and 
dividing by the standard deviation. 

XTIME(I) = Sample times for the input data function in seconds. This 


indexed time is used for plotting routines. 


>- Example 


An excellent example for illustrating many of the features of the FFT 
is given by Stearns (1975). The following example incorporates the same 


essential features. The data function may be written in the continuous 


form as: 


e sint _ t20 
£(t) = (17E.1) 
0 otherwise. 


The Fourier transform of this function is determined from integral tables 


to be: 


o ° 
FCjw) = Ak e “sint e Ia, = = (17E.2) 
8) : (2-W)+ j2w 


The absolute magnitude of the Fourier transform is: 


(17E.3) 


lFcju) | = 
(4+ wy? 


This is the amplitude spectral density estimated by the program. The pro- 


gram prints only the positive frequency values because the transform is 
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Symmetrical. For the reader's interest, the phase function is: 


-2 
8.(w) = ARCTAN[———"—} (17E.4) 
F 2 
(2-w) 

The program does not compute this result because we are assuming interest 
only in the spectral densities. 

The program input is 15 samples of Equation 17E.1 starting at zero and 
going in steps of At=0.3 seconds. The estimated amplitude spectral density 
is shown in Figure 17E-2 and compared with the exact results given by Equa- 


tion 17E.3. The folding frequency for the given sampling rate is: 


1 
f. ~ OAC 1.67 Hz. 


As would be expected, the estimated amplitude spectral density shows the 
effects of aliasing due to components in |FC ju) | beyond the folding frequency. 
Also, the truncation effect of finite sampling introduces further high fre- 
quency components. The estimated amplitude for f=1.46 Hz is 106% too high 
while that for f=0.21 Hz is only 0.3% too high. These effects are common 
in most FFT spectral estimates. 

Figure 17E-3 shows the estimate when the resolution is increased by a 
factor of a? (NRESL=2). It is important to notice that the accuracy of 


the estimate remains unchanged and only the resolution is increased. Figure 


17B-4 shows the estimated power spectral density for the increased resolu- 


tion. 
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Frequency 


Figure 17E-2. Basic Resolution of FFT Estimate 


Frequency 


Figure 17E-3. Increased Resolution for FFT Estimate 
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NRESL = 2 


= ee ee 


Frequency 


Figure 17E-4. FFI Estimate Proportional to 
the Power Spectrum 


Parseval's theorem provides a useful check of the stability and 
accuracy of the estimate and can be used to verify the correctness of the 


program. Using integral tables it is possible to show that: 


t 2 ae 2 
f ce"* sin t) “dt = = f —, = 1/8 volts“/Hz 
0 T le 44 


The integral in the frequency domain can be numerically integrated by the 


following routine to give 


A 7 AQ 2 
AFX (0) +2AF 2 X°(mAF) = 0.1249 volts’ /Hz 
n=1 
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where: 


1 


AF = DEMAX At = 0.208 Hz. 


In the time domain, the result is: 
15 


at = x, = 0.1248 volts”/yz 
teil 


These results may be compared directly with the standard Oia of the 
input data function. These agreements are very good even though aliasing 
has produced an estimate with considerable error near the folding frequency. 

' Printed along with the amplitude spectral density function are the 
absolute magnitudes of the Fourier series components that would be used to 
represent the input data function. These amplitudes are related to the 


density function by the equation: 
A A 
c, = AFX (nF) 


Refer also to Equation 11.13. 
An example of the effects of white Gaussian noise on the estimate is 
given in Chapter XIX and a detailed discussion presented in Chapter XII, 


Section D. 


F. PROGRAM 06 - Estimation of Single-Sinewave Parameters 


1. Introduction 
This program implements a digital Fourier transform amplitude spectral 
estimate for 2 single frequency component with a frequency that may be con- 


tinuously varied. The input is assumed to be a single-frequency sinewave 
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plus bandlimited Gaussian noise. For a selected frequency below the Nyquist 
frequency, it will compute a "best" fit. The fit is "best" in the sense 
that the amplitude and phase represent an orthogonal component of a discrete 
Fourier series which can represent the input data function. 
+ 

The estimate is improved if the input data is first preprocessed to 
remove any linear trends. To guarantee a zero-mean data function, the pro- 
gram computes the mean value and removes it before doing the spectral esti- 
mate. The actual algorithms are based on Equations 13.13 through 13.18. 
These results will be presented here in a slightly different manner to pro- 
vide a better understanding of the estimator. 


A continuous complex amplitude estimate (not an amplitude density) is 


defined as 


2 
-j = Kat 
- 1 +N To 
G(£,) = NGL x x(kAtje (17F.1) 
=-N 


where At=T,/(2N+1). Discrete coefficients are defined at the frequencies 
corresponding to f,=n/Ty- In general, the continuous complex amplitude is 
defined for any frequency between zero.and the Nyquist frequency. 


If em) is represented in complex notation 
C¢£,) = aC £,) + 58(£,) | (17F.2) 
the complex components can be computed from the following algorithms: 


i 2N+1 on 
at £)) x= x(kAt) cos Ca (k-N-1)At] (17F.3) 
° 


+1 k=1 


B 


_y (2 toate 
8S Seay a x(kAt) sin Se sa Lat] (17F .4) 
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This method of defining the summation index forces the phase angle to be 
referenced to the center of the input time series. From Fourier analysis 
(Appendix II), the complex amplitudes are related to the usual a and b co- 


efficients as follows (Equation A2.13): 
=C +c" 
Ca ne a (17F.5) 
b_ = j(c c* 
pes) (17F.6) 
For the definition of continuous estimates this becomes: 


a(£,) =2a(£,) (17F.7) 


b(£,) = -2 8 (£,) (17F.8) 
The absolute magnitude of a complex amplitude estimate is: 
Jace.) | = Coe. + (eT = $ face +b ce) FF (178.9) 


The absolute magnitude of a real frequency component is the sum of both 


positive and negative frequency contributions: 
ace.) = 2{écz,)| (17F.10) 
The phase angle for the complex component is 


BCE.) . 
6,(£,) = aRcrAN [> zy! (17F.11) 


while for the real component it is: 


$4(£,) = ,(£,) - 270° (17F.12) 
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The terminology "real" and "complex" refer to the observable sinusoid and 
its double-sided mathematical representation. Finally the "residue" time 
series is defined as: 
x, (kt) = x(kAt) - d(£,) sin (7 (kw -1)at] (17F.13) 
° 

The program output prints the input parameters and the input data funce- 
tion Statistics. The input time series and the zero-mean time series are 
also printed. Next, the magnitude, frequency, and phase of the best-fit 
sinewave are given. The "residue" time series is computed and its statis- 
tics are printed. Finally, the zero-mean input, sampled sinewave, and 
residue are printed. 

It should be emphasized that this technique is not superior to doing a 
complete FFT on the input data. It has advantages in its simplicity and 
selectable frequency but the FFT gives an estimate of all components as 


compared to only one. 


2. Input/output data definitions 


CMAG = Absolute magnitude of the complex coefficient defined in 


Equation 11.15 and evaluated at the chosen frequency. 


CPHASE = Phase angle of the complex coefficient in degrees. 

DMAG = Amplitude of the "best-fit' sinewave. 

DPHASE = Phase angle of the "best-fit" sinewave in degrees referenced 
to the center of the input time series. 

FREQ = Frequency in Hertz at which the "best-fit" sinewave is 


evaluated. 


FS(I) = Sinewave estimate evaluated at the sampling times. 
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NS = The total number of time samples in the input data function. 

SNAME = A name or label in FORMAT (5A4) used to identify the data 
record, 

TDELT = Sampling interval for the input time series, seconds. 

TZERO = Period for which the sinewave approximation is evaluated. 

X(I) = Input time series data. 


XMDC(I) = The input time series eh the average value removed to give 
a zero-mean time series. 

XR(I) = This is a time series generated by subtracting the sinewave 
from the zero-mean input data. It is referred to as the 


"Residue" time series. 


3. Input/output statistics 


DCV = Average value or estimated mean of the time series. 
STDEV = Standard deviation. 
XBAR2 = Mean-square value. 
XVAR = Unbiased estimate of the variance. 
4. Example 


The time series data for this example is the same as used for PROGRAM 
04 except a linear regression analysis has been performed to remove any 
linear trend. The first printout, the input parameters, shows the total 
number of samples (NS=41), the period of the desired spectral component 
(T, = 49.26 seconds), and the sampling interval (At = 60 seconds). The sta- 
tistics for the input data function are computed and printed to be used for 


comparison with final results. For this example, the input time series 
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and zero-mean time series are equal because the data was previously processed 
to remove the mean value. 


The next output is the desired "best-fit" sinewave describing the in- 


put time series 


f, = 128.46 sin(S t- 1.839) (17F.14) 
° 


where: 
T =4.926x 102 seconds. 
re) 
Frequency = 3- = 2.03 x 107? Hz. 


° 
Phase = -105.34 degrees 


A statistical measure of the quality of this estimate can be obtained 
by applying Rayleigh statistics as discussed in Chapter XIX, Section C. 


First we assume that the time series is bandlimited Gaussian noise and test 
the hypothesis that the output amplitude spectral density was generated 


exclusively by noise. The expected value of the Rayleigh output is com- 
puted as: 


Ett £)} ~ at as x2) (17F.15) 


For At=60, NS=41, and ee 1.4904 x 10°, the expected value becomes: 
A 
E{X(£) } = 4.1566 x 10° = @*o, (17F.16) 
The estimated sinewave amplitude is converted to an equivalent amplitude 


Spectral density by: 


A 
XK, = NS At Pes, (17F.17) 
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For DMAG=128.5, the estimated amplitude density becomes: 


A 4 
x, ™ 15.8055 x 10 


Since we know that og, = 3.3165 x 10°, the estimated amplitude is at the 4.766 
sigma-level on the Rayleigh distribution. The probability that an a priori 


selected sample at this frequency could be due to random noise is: 
A 1 2 -5 
P,=P(K>4.766 o) = Exp[- > (4-766) J=1.17x10 (17F.18) 


This is an extremely low probability. The probability that any amplitude 
estimated by a Fourier transform would exceed this value with only noise 
input is given by: 


4 


A NS = 
P,=l-(1-P,) = 4.789 x10 (17F.19) 


This is also a low probability. A more detailed discussion of how to in- 
terpret estimated spectra is given in Chapter XIX, Section C. 

To give the reader a relative idea of the quality of the sinewave fit, 
the zero-mean data function, "best-fit" sinewave, and "residue" time series 


are plotted together in Figure 17F-1. 
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INPUT TIME SERIES 


time 


2 "BEST FIT' SINEWAVE 


time 


"RESIDUE' TIME SERIES 


time 


Figure 17F-1. Single-sinewave Parameter Estimation 
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G. PROGRAM 07 - Maximum Entropy Spectral Analysis 


1. Introduction 

This program computes the maximum entropy estimate of the power spec- 
tral density function of a finite time series. A finite autoregressive 
series is used to model the input data function and the power spectrum is 
computed using the autoregressive spectral estimator given by Equation 
15F.1. The estimates of the autoregressive coefficients are obtained from 
the Burg MEM recursion algorithm described in Chapter XV, Section G. 

The program is intended to be used for routine data processing and has | 
been user oriented as much as possible. Processing features include user 
selected spectral output display resolution and autoregressive order. A 
selectable printing option has been included so that routine or unwanted 
output can be suppressed. Selectable data preprocessing options include 
removing the average value, linear trend removal, and converting the input 
to standard variables. 

Printed output inciudes the input data parameters, input time series, 
data function statistics before and after linear regression, time series 
after linear trend removal, standard variables, MEM starting values, inter- 
mediate coefficients (optional), summary of iterative results, maximum 
filter length, error power, estimated autoregressive coefficients, peak 
value of the estimated spectrum, total spectral power, and the normalized 
power spectral density estimate. 

The folding frequency is computed from the input data and the program 
inhibits the calculation or printing of any aliased output. The spectrum 


is scaled such that the output is the positive frequency half of a double- 
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sided spectrum. Since the spectral power has been normalized to unity, 

the program integrates the spectral density to obtain the total power in 

the estimated spectrum which may then be compared with unity. This provides 
a reasonable check on the quality of the estimate and the numerical stabil- 
ity of the algorithms. 

SUBROUTINE ATOREG recursively computes estimates of the autoregressive 
coefficients using the Berg algorithm. A detailed description of the rou- 
tine is given in the flow diagram of Figure 17G-1. The equation numbers 
refer to those equations in the text which are implemented in the sub- 
routine. 

SUBROUTINE MEMSPM computes the power spectral density estimate using 
the autoregressive coefficients from ATOREG. This algorithm implements 
Equation 15F.1. 

A simple flow diagram describing the essential features of the entire 


program is given in Figures 17G=-2a and 17G-2b. 
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SUBROUTINE ATOREG 


T=1,N 
P=P+X(T)**2 


PZERO = P/N 


K=1 
B1(1) = X(1) 
B2(N - 1) = X(N) 


T=2,N-1 
B1(T) = B2(T - 1) = X(T) 


XNOM = XDEN = 0. 


T=1,N-K 
XNOM = XNOM + B1(T)*B2(T) 
XDEN = XDEN + B1(T)**2 + B2(T)**2 
Al(1) = 2. *XNOM/XDEN 
P(1) = PZERO*(1.-A1(1) *A1(1)) 
PRINT ,A1(1) ,P(1) ,P(0) 


T=1,K-1 
AA(T) = AK(T) 

T=1,N-K 

B1(T) = B1(T) - AA(K - 1) *B2(T) 

B2(T) = B2(T+ 1)-AA(K - 1)*B1(T +1) 


XNOM = XDEN = 0. 
T=1,N-K 


XNOM = XNOM+ B1(T)*B2(T) 
XDEN = XDEN + B1(T)¥**2 + B2(T)**2 


P(K) = P(K- 1)*(1.- AK(K)**2) 


t 
4 
v 


Figure 17G-1. 


The mean-square value is computed 
in the main program. (Equation 
15G.5) 


Compute starting values for Bl and 
B2. (Equation 15G.19) 


Compute the K=1 values of AK(K) 
and P(K). (Equation 15¢.12) 


(Equation 15G.4) 


Iterate 
Increment the order K. 


Compute the autoregressive coeffi- 
cients AK(T) using recursive equa- 
tions for increasing values of K up 
to K=MM. (Equations 15G.10, 
15G.11, 15G.15, 15G¢.16) 


Compute the highest order coeffi- 
cient AK(K) for a Kth order process. 
(Equation 15G.12) 


Compute the mean-square prediction 
error. (Equation 15G.4) 


Burg MEM Recursion Algorithm for Computing 


Estimates of the Autoregressive Coefficients 
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SUBROUTINE ATOREG (CONT.) 


T=1,K-1 Compute the estimated autoregres~ 
AK(T) = AA(T) - AK(K) *AA(K - T) Sive coefficients fer a Kth order 
PRINT, K, T, AK(K) model of the process. Printing of 

intermediate values is optional. 

(Equation 156.3) 


—- -—- Iterate 


Print a summary of the iterative 
results for increasing values of K. 


Print the final order and corres- 
ponding error power. Print all of 
the autoregressive coefficients 
for a Kth order estimate. 


PRINT, T, AK(T) 


Figure 17G-1. (Continued) 
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"USER' CHECK READ INPUT COMPUTE 
DIMENSIONS PARAMETERS ve 
COMPUTE PRINT READ INPUT 
AVERAGE VALUE INPUT PARAMETERS DATA FUNCTION 

COMPUTE COMPUTE PRINT 
LINEAR STATISTICS a eres 
REGRESSION 


& NC 


NO 
TNREG = 2 Re. 
I) 


YES 
CALL 
X(I) = XSV(Z) SUBROUTINE SUBROUTINE 
ATOREG  - MEMSPM 
<=) ‘USER' SUPPLIED 
YES ; 


ADDITIONAL RUN 


Figure 17G-2a. Main Program Flow Diagram. 
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SUBROUTINE ATOREG 
‘USER’ CHECK COMPUTE pa ie 
AUTOREGRESS IVE Lapis 
DIMENSIONS STARTING VALUES 


"USER’ CHECK COMPUTE MAXIMUM 
MEM SPECTRAL 
Ee DIMENSIONS FREQUENCY INDEX COMPONENTS 


SELECT THE 
LARGRST SPECTRAL 
COMPONENT 


Figure 17G-2b. Subroutine Flow Diagrems 
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2. Input data definitions 


FDELT = Output frequency display resolution of the MEM spectral 
estimate in Hz. If the user selects FDELT=0., the program 
will default to a value equal to four times the Fourier 
resolution. 

LINREG = A user selected data preprocessing option. Spectral esti- 
mation may be performed using XSV(I), XNORM(I), XMDC(I) or 
X(I) by choosing LINREG= 3, 2, 1,0 respectively. 

MM = User selected value for the maximum order of the MEM model. 
This is the maximum number of autoregressive coefficients 


used to characterize the input time series. 


NS = Total number of input data samples. User must dimension 
program variables accordingly. 
PIOPI = A printing option used to control the printed output of the 


program and used to suppress unwanted output. PTOPT is a 
binary word of the form: (NPT1, NPT2, NPT3, NPT4). NPT1 
controls the printing of the input data function and standard 
variables. NPT2 controls the printing of the summary of 
iterative MEM results. NPT3 controls the printing of the 
autoregressive estimation summary. NPT4 controls the print- 
ing of the MEM normalized spectral density. 

SNAME = A name or label in FORMAT (5A4) used to identify the data 
record. 

XIDELT = Sampling interval for the input time series, seconds. 

X(I) = Input time series data. Units of volts will be assumed for 


Simplified presentation. 
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3. Input data function statistics 


x = XDC = Average value or estimated mean value of the data function, 


volts. 
2 = PT = Mean-square value of the input data function, volts’. 
on = PZERO3 = Unbiased estimate of the variance of the input data 
function, volts . 
o, = STDEV3 = Standard deviation, volts. 


4. Linear regression 


bo = BZERO = Intercept point from linear regression. 

by = BX1 = Slope from linear regression 

on = PZERO2 = Variance of the data function after removal of the linear 
trend. 

Oo, = STDEV2 = Standard deviation. 


’ 


5. Output data definitions 


Al(1) = Starting value of the autoregressive coefficients for the 


MEM recursion algorithm. 


AK(K) = K-th autoregressive coefficient for an MM-th order process. 

FDELT = Output resolution of the MEM spectral estimate, Hz. 

FR(I) = Frequency points at which the spectral estimate is computed, 
Hz. 

K= - Index for the autoregressive coefficients. 

L= Index for spectral components. 

NMAX = Maximum spectral index number computed on the basis of 


aliasing. 
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P(O) = Normalized total power for the time series (equal to the 
variance). 

PC(1) = Starting value of the mean-square estimation error for the 
MEM recursion algorithm. 

P(K) = Mean-Square estimation error for the K-th order autoregres- 
sive estimate. 

PT = Total power (or variance) for the data function after linear 
regression. 

S(I) = MEM estimate of the output power spectral density (or second- 
moment density) function, watts/Hz. The printout gives the 
positive frequency half of the double-sided density function. 

SBIG = Largest spectral component in the estimated spectrum. 


SPLOT(I) =The relative power spectral density with respect to the 
largest component. 

XMDC(I) = Input data function with the average value removed. 

XNORM(I) = Data function after linear regression. 

XSV(I) = The input data function converted to standard variables after 
linear trend removal. 


XTIME = Sample times for the input data function, seconds. This 


indexed time is used for plotting routines. 


6. Example 
The time series data in PROGRAM 04 will also be used for this example 


because it provides a demonstration of the resolution differences between 
the DFT and MEM. A 9th order autoregressive estimate was chosen for the 41- 


point time series and the selected output resolution was 2.5 x 107? Hz. This 
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is approximately four times more resolution than the default value in the 
program. Ali the printing options were chosen so that the output features 
could be demonstrated. 

Since the raw data exhibits a pronounced average value as well as a 
linear trend, it was decided that the estimated spectrum would be computed 
on XSV(I), the input data function after removing the linear trend and con- 
verting to standard variables. The input data before and after removing 
the linear trend is shown in Figure 17G-3. The time series after pre- 
processing is shown in Figure 17G-4. This is the series for which the spec- 
trum will be estimated. Notice from the data function statistics after 
linear regression that the variance has been reduced to approximately 61% 
of its previous value. 

The MEM starting values are printed so the user may see the total 
power (mean-square value) in the data function and the error power in a 
first order estimate. A summary of iterative results is printed to illus- 
trate the trends in the reduction of the total error power and the extra- 
polated coefficient, {AK(K): K=1, 2, ... MM}. The mean-square estimation 
error follows the typical trend of decreasing rapidly with order until some 
minimum level is reached. Figure 17G-5 illustrates this effect. 

The autoregressive estimation summary gives the mean-square estimation 
. error for the chosen order (MM=9) of the estimate and the best-fit auto-- 
regressive coefficients. The error power indicates a white noise contribu- 


tion of only 8.6% to the total spectral power. 
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The maximum entropy power spectral density estimation output prints 
the spectral density in watts/Hz as well as relative amplitudes in dB. The 
numerical integration to obtain total power produces a result very close to 
unity and this indicates good numerical stability. The largest spectral 
component occurs at a frequency of 1.925 x107> Hz. Plots of the power spec- 
trum in watts/Hz and in relative dB are shown in Figures 17G-6 and 17G-7. 

These results should be contrasted with the DFT estimate given by 
PROGRAM 04. Comparing Figures 17D-2 and 17G-6 we see the spectral peak is 
much sharper for MEM. Also the fringe frequency estimate given by MEM is 


2 Hz.) than that given by the 


mich closer to the true frequency (1.95x 10° 
DFT. A further discussion of MEM resolution and this example is given in 


Chapter XIX. 
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Figure 17G-5. Estimation Error as a Function of 
the Order of the Estimate 
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XVIII. MAXIMUM ENTROPY AND FOURIER TRANSFORMS 


A. Introduction 

This chapter is devoted to a summary of the properties, similarities, 
and differences in DFT and MEM spectral estimation. These techniques are 
emphasized because they represent two of the most basic philosophies on spec- 
tral estimation: periodic extension in the time domain (DFT) and extrapola- 
tion in the autocorrelation domain (MEM). Throughout the presentation we 
will keep in mind the fact that spectral analysis is usually employed to 
detect periodicities in a data function and to determine the "strength" and 
"realness" of the estimated component. This presentation is by no means 
exhaustive in its coverage nor does it present the variety of viewpoints 
that are possible. Also, much of the material is redundant in the fact that 


it may be found in various forms elsewhere in the text. 


B. Fourier Transform Spectrum Summary 
Since the DFT is a discrete implementation of the Fourier transform it 
is helpful to discuss these properties first. A time function {x(t):-°<ts 
+} and its amplitude spectral density function {X(ju):-© Ss ws+o} ‘form a 


Fourier transform pair (w= 2mf): 


+o -5 

X(ju) =f x(t) JE ae (188. 1) 
+o , 

x(t) =f X(jwe ae (18B.2) 


The Fourier transform does not exist for all possible types of data func- 
tions, a good example being white noise. If the Fourier transform is to 


exist, it must be piecewise continuous on every finite interval (Riemann 
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integrability) and it must have a finite average value (necessary). A suf- 
ficient (but not necessary) condition is that it be absolutely integrable. 

Examples of functions which are most easily analyzed by Fourier trans- 
form techniques are single pulse waveforms and time functions modified by a 
finite data window. These types of transforms are commonly determined both 
analytically and numerically. 

The Fourier transform has the following useful properties: 

1. The real part of X(jw) is an even function in wy; Re{X( ju) } = 

Re{X(-ju) }. 

2. The imaginary part of X(jw) is an odd function; Im{X(juw) } = 

~Im{X(-ju) }. 
3. The amplitude spectrum is an even function; Ixcju) | = [xc-juy |. 
Although not specifically a property, the majority of the functions analyzed 
using the Fourier transform are time limited: x(t) =0 for e| ee aw This 
produces an amplitude spectrum of infinite extent. The absolute magna eade 
squared, Ixcjup [7, of the amplitude spectral density function is called the 
energy spectrum of x(t). This concept is used to relate time and frequency 
in the definition of Parseval's theorem: 
+0 +c 
$ txciw Pat= fo fxcey [Pat (188.3) 
aa) =o 

The energy spectrum of white noise does not exist because the Fourier trans- 


form is undefined. 
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C. DFT Spectrum Summary 
A discrete Fourier transform pair can be obtained from Equations 188.1 
and 18B.2 by using a Riemann sum approximation for the integrals. Fora 
sampled time function, {x(kAt): k=-N,-N+1,...,0,...,N-1,N}, containing 
(2N+1) samples of an infinite time series, the discrete Fourier transform 
(DFT) is defined by: 
aN 


X(jnaw =At I x(katye JRAdkdt (18¢.1) 
k=-N 
Aw bl +jnAukAt 
x(kat) =5= EZ X(jndwe J (18¢.2) 


=-N 
There is absolutely no restrictions on the type of function used to obtain 
a DFT. However, if the DFT is used to make an estimate of the amplitude 
spectral density function cr energy spectrum of a continuous time function, 
x(t), the quality of the estimate will depend not only on the "true" proper- 
ties of x(t) but also on the limited ability of x(kAt) to faithfully repre- 
sent x(t). The commonly used estimates for X(jw) and x(t) in terms of the 


sample function x(kAt) and its DFT are: 


+N 


X(ju) =at EF  x(katye Jue (180.3) 
k==-N 

A Aw = +jnAwt 

x(t) = £2 5 xCjnawe (18¢.4) 


The DFT has the following properties: 
1. The real part of X{jnduw) is even. 
2. The imaginary part of X(jnAw) is odd. 


3. The discrete amplitude spectrun, Ix jnav) |, is even. 
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4. The frequency interval of the estimate is: Aw=2n/(2N+1)At. 

From the sample function x(kAt), the estimated Fourier series is com- 
puted using (18C.4) and has the following properties: 

l. X(t) =x(kat): t=0, +At, +2At, ..., +NAt. 

23 *(t) is periodic with period, Ty = (N+ 1) at. 
The amplitude spectral density computed using (18C.3) has the following 
properties: 

l. It is periodic with period 27/At, X(jw+ j2n/At) =X(ju). 

2. £¢(ju) =x" (-ju). 

3. The estimated amplitude spectrum is even, Ic guy | = [X¢-50y |, and 

is uniquely evaluated in the range, OS wsn/At. 


Parceval's theorem for the DFT is given by: 


iy 2 1 oy | 2 
At 2 x (kat) =a Fs [xCjnaw) | (18C.5) 
near (2N+1)At Ws 


An interesting property of the DFT concerns the concatenation of zero's 
to the data function. The new data function defined by 


y(kAt) = x(kAt) {k=0,+1, +2, ..., +N} 
(18C. 6) 


=0 {k=+(N+1), ..., +4} 
improves the resolution (not "“accuracy") of X(jnAw) by decreasing Aw. This 
artifice produces a more continuous and hence pleasing display of the 
result. For a single time pulse there is no loss of accuracy but for a 
truly periodic function the period becomes distorted and an additional 


estimation error is introduced. 
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D. Wiener-Khinchine Relations 

The concept of a power spectral density function (as opposed to the 
amplitude spectrum) is based on the definition of a time autocorrelation 
function for a wide-sense stationary random process. The autocorrelation 
function and the power spectral density function form a Fourier transform 
pair called the Wiener-Khinchine relations. The advantage of this concept 
is that the autocorrelation and power spectrum are defined for both periodic 
and random components. A disadvantage is that more numerical computation 
is required than for the DFT. The power spectrum of a random process is 
presented in Appendix I, Part F and in various other chapters. 

Applications of the Wiener-Khinchine relations include the Blackman- 
Tukey method, the maximum entropy method, and the filtering method of spec- 


tral analysis. An important model for all of this methods is discussed in 
Chapter V. Most important are the time domain convolution integral (Equation 


5A.1), the autocorrelation domain convolution integral (Equation 5B.5), and 
the linear filter power spectrum relationship (Equation 5B.8). 


E. MEM Spectral Theory Summary 

Any process with a bandlimited, finite, and continuous power spectrum 
can be represented by an infinite moving average (Equation 14.1). The fil- 
tering analogy of a moving average process is the linear transformation of 
white noise. The time series at the output of the filter can be represented 
arbitrarily close by a K-th order (K possibly infinite) autoregressive series 
(Equation 14.21). The maximum entropy method maximizes the entropy of the 
spectrum (Equation 15D.1) subject to the constraint that the inverse Fourier 


transform of the bandlimited power spectral density function (Equation 15F.1) 


315 


for a finite number of sampled autocorrelations be exact (Equation 15D.10). 
This is equivalent to a least-square error fitting of the time series with a 
finite number of autoregressive coefficients. Given enough data, this fit 


can be made arbitrarily close by increasing the order. 


F. MEM and DFT Comparisons 

The biggest contrast between DFT and MEM spectral estimators is that the 
former is an amplitude spectrum and the latter a power spectrum. They also 
differ in the fundamental way they approach the theoretical need for infinite 
data. The DFT assumes a periodic xteneion- be the sampled data function, 
x(kAt), while MEM estimates an extrapolated aitbebeteie ion function based 
on an autoregressive model. Practical applications have shown that extra- 
polation results in an estimator with superior resolving power. Increases 
by a factor of 4-5 over DFT have been reported in the literature (Ulrych 
et.el., 1973). 

For bandlimited random processes, the power spectrum estimate of MEM 
converges with increasing amounts of data while the DFT estimate does not. 
The spectral amplitudes for the DFT estimate, IX¢jnaw) |, are always Rayleigh 
distributed for Gaussian noise no matter what the record length. 

The DFT spectral estimate is periodic, bandlimited, and discrete be- 
cause of the periodic extension assumption and the Nyquist sampling theorem. 
The MEM spectral estimate is periodic, bandlimited, and continuous because 
of the autoregressive model. Both can be designed to give an estimate that 
is continuous as frequency although practical resolution is usually limited 


by the required computing effort. 
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The DFT estimate is “optimm" in the sense that x(t) (Equation 18C.4) 
exactly reproduces the original time series, x(kAt), at the points t=kAt. 


The mean-square error of the time domain representation is thus constrained 


to zero: 
E{[x(kat) - x(kAt) ag =0 (18F.1) 


Because the estimator was also constrained in frequency (bandlimited) it 
also provides the "smoothest" extrapolation between data points, x(kAt), 
consistent with the restriction that x(t) be periodic. The accuracy with 
which the spectral estimate, X(jnAw), agrees with the "true" spectrum, 
X(ju) ="P{x(t)}, depends upon how well x(t) is modeled by a bandlimited 
periodic function. 

The MEM estimate is "optimum" in the sense that the spectral entropy 
(Equation 15D.1) is minimized while constraining the estimated autocorrela- 
tion function to be the exact inverse Fourier transform of the power spec- 
tral density. The mean-square error in the time domain is minimized by 


choosing autoregressive coefficients which minimize (Equation 13.36): 
A 2 2 
E{{x(kat) - x(kat) }°} = of (18F .2) 


An interesting parallel between the DFT and MEM can be developed using 
ideas presented in Chapter XIII. Equation 13.9 represented the Fourier 
series in a form similar to autoregression. The autoregressive approach 
starts with the best linear one-step predictor in a least-squares sense: 

K 


K(k+1)= 5 a x(k+ 1-2) (187.3) 
n=1 ‘ 
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Using a complex representation for a sinusoid of frequency, f£, we have: 


+j2nfkaAt 
x(k) =Ae (18F.4) 


than 


If, ideally, the prediction error is to be zero, x(k+1) -x(k+1 =0, then 
substituting Equation 18F.4 into 18F.3 gives the condition for the complex 


coefficients as: 


. K : 
ge 1TECREL) AE ; ave area: (18F.5) 
: n=l 
This can also be written as: 
= ~j2nfnat 
l= = ae (18F .6) 
n=1 a 


The solution for an from (18F.5) is given as: 


_ iL +j2mindt 
a,7*xe (18F.7) 


These are the complex coefficients needed to perfectly predict (or filter) 
the single-frequency sinusoid. 


For more than one sinusoid the solution is much more involved. Consider 


the representation 
x(k)= EZ Ave P (18F .8) 


where a The condition for the complex coefficients now becomes: 


M +jA_ (k+l) K M +jA_(kt+l-n) 
>» Ale P = 2a Ae ?P 


(18F.9) 
p=1 


The solution for a, is given by 
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+jd_ (k+l) M +jX_(ktl-n) 71 
P [= Ae P 7 (18F.10) 


1 M 

a ==>[Z Ae 
K 

p=1 


p=1 P 
which reduces to (18F.7) for M=1. Notice the k dependence of a when more 
one frequency is involved. This means that ay depends on time which is not 
allowed for autoregressive coefficients. In conclusion, MEM cannot repre- 
sent the Fourier series given by (18F.8). These representations may be con- 
trasted with the Fourier series representation of Equations 11.9 and 11.10 


to determine further comparisons. 
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XIX. INTERPRETATION OF ESTIMATED SPECTRA 


A. Introduction 

A study of the properties of the various spectral estimators is only 
part of the task necessary for the seneeicall application of spectral theory 
in research. Even after careful selection and implementation of a good es- 
timator one is faced with the task of interpretation of the computed results. 
In this chapter we will concentrate on the DFT (FFT) and MEM techniques. 
The important properties that affect the interpretation of the results of 
these estimates are presented along with practical examples. Two of the 
most important concepts presented are the detection criterion for a sine- 
wave in Gaussian noise using the FFT and the effects of data length and 
autoregressive order on the estimated frequency for MEM. The summaries on 
resolution limits and common estimation errors should be especially helpful 


for designing the data taking and processing methods. 


B. Data Function Models 
It is helpful in the interpretation of estimated spectra to keep in 
mind a suitable model for the process. Three possible models were suggested 
in Chapter XIII that seem to be able to describe most empirical data func- 


tions. These models are: 


1. Fourier series plus bandlimited Gaussian noise. 


. 2™ 
ay OT ne £ 
y(t) =§(t) + Big ce (19B.1) 
n=]. 


2. Autoregressive series of order K. 
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K 
y(k) = §(k) + z a x(k - n) (19B.2) 
n= 


3. Convolution of a known function and bandlimited white Gaussian noise. 
y(k) = z(k) * n(k) (19B.3) 


Strictly speaking, the amplitude spectral density does not exist for 
these models because they de not have Fourier transforms. As a practical 
matter however only a limited time record is ever observed and thus the 
actual empirical data model is the product of y(t) and a suitable time do- 


main window: 


x(t) = y(t) + w(t) a (19B.4) 
w(t) #0 le se, (19B.5) 
=0 otherwise 


The Fourier transform of x(t) exists for all of the suggested models. The 
effect of the window function and random noise on the estimated amplitude 
spectrum are a topic of discussion in this chapter. 

The power spectral density function exists for all the models of y(t) 
because they all have well-defined autocorrelation functions. The “unwin- 
dowed" power spectrum is usually referred to as the "true" spectrum and 
‘the quality of the estimate is judged against the "true". This is not a 
really fair comparison in many cases because of the degrading effects of 
data windowing. Whenever possible, an estimate should be compared to the 


spectrum of the "windowed" function. 
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For purposes of reference, the power spectral density functions of 
the models will now be given. For all cases, the spectral models will be 


obtained from the following: 


E{y(t) y (t- t)}=RL(7) (19B. 6) 
PR (7) = 8 (£) (198.7) 
FH felE(c) § (Tt) }}=S.(£) (198.8) 


The spectra of the models are: 

1. Fourier series plus bandlimited Gaussian noise. 
+M 
x 


» 2 yg 
S(£) =S_(£) + le | nb £) (198.9) 


n=-M 


2. Autoregressive series of order K. 


ra 


| s-(£) 
$(£) = ————— —_____— (198.10) 


i 
S_(f) =o, At [el sf, (198.11) 
= otherwise 
1 
fy = Dae 


3. Convolution of a known function and bandlimited white Gaussian 


noise. 


sy) = S (£8 _(£) (19B.12) 
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s_(£) =H( ju) (ja) (convolution filter) (198.13) 
2 
$.(£) =o- At el< : 
e(£) =o, A lel se, (19B. 14) 
=0 otherwise 
= 
ty * Dat 


There are many pathological functions which do not fit these models 
and whose spectra are not well-estimated but for the majority of noisy 
data, these models are adequate. Also, there seems to be a reluctance on 
the part of some theorists to accept the noise model restrictions of finite 
bandwidth and Gaussian distribution on the noise model. Bandlimiting is 
not a problem either conceptually or practically as all measurement tech- 
niques cause bandlimiting. The Gaussian assumption is harder to justify 
conceptually but for practical applications the Central Limit Theorem 
deuslly appties because of ali the linear operations associated with signal 
processing. In most commmication and signal processing systems the narrow- 


band filtering (linear filtering) generates Gaussian noise. 


C. DFT and Gaussian Noise 
The DFT is still commonly used for analyzing noisy or almost random 
data in spite of the fact that nearly every book on signal processing cau- 
cine that the DFT estimate for random noise does not converge. This mis- 
use is probably a result of the following circumstances: 
1. The FFT is readily available and the Fourier transform method of 


estimating an amplitude spectrum is widely understood. 
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2. The subocoreelarion method of estimating a power spectrum is more 
difficult a the Blackman-Tukey approach is not as widely under- 
stood. 

3. The data window effect (Equation 19B.4) allows the existence of a 
DFT for x(t). It also allows for a statistical interpretation of 


the estimate and for spectral smoothing techniques. 


The application of the DFT to a limited time record of bandlimited Gaussian 
4 


noise will now be presented. 
‘The complex Fourier amplitudes for a random signal defined by {E(k): 


k=0,+1,+2,...,+4N} are given by the complex series: 


. _2TT 
+N -j nk © 
Be ek 2N+1 
C.-L nai E(k)e (19¢c.1) 


The sampling interval in time is At and is represented at times, t=kAt, 


by the index k. An alternate representation for (19C.1) is given by 


c. = = (& + 58.) (19¢.2) 
where 
| & = &(0 © CeCe) + k)] cL 19¢.3 
= (0) + E [EGG + §(-k) Joos GIy ak) (19¢.3) 
A x 21 
B= © [S(k) - &(-k)]sin (@A™ nk) (19¢.4) 
¢ =¢ (19¢.5) 
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A 
& and 8. are both Gaussian because the operations described by (19C.3) and 


n 
(19C.4) are just scaling and linear combinations of the Gaussian random 
variables, €(k). Even when §(k) is not Gaussian the central limit theorem 
applies and the linear combinations of many samples makes a -and 5 tend 
to be Gaussian. 


The discrete amplitude spectrum is estimated by scaling the absolute 


value of the complex Fourier amplitudes (AE = 1/(2N +1) At): 
A A 
[Xcmat)| = (2N+ 1)at]e, | (19¢.6) 
A 
In terms of the om and a the estimated amplitude spectrum is: 
A A 
[Ecaae) | = acca? + $2)? (19¢.7) 


From statistical commumication theory (Whalen, 1971) we know that X(nA£) 
represents an envelope function and hence is Rayleigh distributed. Since 
the Rayleigh characteristic does not depend upon the number of samples used 
in the estimate, the amplitudes remain Rayleigh distributed no matter how 
much data is used. For this reason the amplitude spectral estimate is said 
to be nonconvergent. 

Figures 19C-1 ana 19C-2 show the estimated amplitude spectra for sample 
records of a bandlimited white Gaussian noise process. The "parent" Gaussian 
process has the following properties: 

1. Mean: E{E(k)}=0. 

2. Mean-Square: E{€7(k) }=1. 

3. Variance: E{(E(k) - E{E(k) })7}=1. 


4 Power Spectrum: S-(£) =1/At, \e| sf: 
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The first sample data record with M= 32 points has the following properties: 
1. Mean: <E(k)>=3.25 x10" 
2. Mean-Square: <"(k)>= 1.39 
3. Variance: <(E(k) - <&(k)>)?>= 1.43 
The nonnegative half of the estimated amplitude spectrum shown in Figure 
19C*l has the following properties {n=0,1,2,..., g ~- 1)}: 
1. Spectrum: Rayleigh distributed 
2. Mean: <|X (AE) |>= 5.94 
3. Mean-Square: <|X(na£) |2> = 43.8 
4. Variance: <( IX (nae) | - <{X(na£) \>) co 9.06 
The second sample record with M=256 points has the following properties: 
1. Mean: <€(k)>=-1.52 x1077 
2. Mean-Square: <t2(K)>= 0.979 
3. Variance: <(E(k) - <€(k)>)2>= 0.982 
The nonnegative half of the estimated amplitude spectrum shown in Figure 
19C-2 has the following properties {n=0,1,2,..., & - 1)}: 
1. Spectrum: Rayleigh distributed 
2. Mean: <|X(na£) [>= 13.8 
3. Mean-Square: <|X(aa£) hee 250. 
4. Variance: <( [Xcnae) | -<X(nas) [>> = 61.4 
Parceval's theorem for the DFT as given in (18C.5) can be rewritten in 


terms of the present quantities to be: 
A as A 2. A 
gem <t7(k)> = ala <|X(aas) [> - [X(0) [73 (19¢.8) 


For At=1 and Af=1/MAt the estimates above (for M= 256) give 


A 


X(ndf), Amplitude Spectral Density 


32,00 40,00 


24.00 


18.00 


8.00 
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AMPLITUDE SPECTRAL DENSITY FUNCTION 


nAf, Frequency 


Figure 19C-1. Estimated Amplitude Spectral 
Density for Gaussian Noise (M= 32) 


M 
S-) 
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At u<t2(K)> = 251. 
and: 


afl M<|X (mAs) [2>~ |X (0) [2] = 250. 


This is in excellent agreement. We also know from theory that, for a truly 


Rayleigh distribution, the mean and mean-square values should be related as: 
e{ [X¢mae) |7}= + 57f |X cane |} ; (19¢.9) 
For estimates, the approximation is: 
<|X(na£) (> ~= <{Rcaae) [>2 (19C.10) 


For the estimate shown in Figure 19C-2 the ratio of right to left-hand-side 
of (19C.10) is 0.970 which is a three-percent difference. 

Combining Equations 19C.10 and 19C.8 gives an important result that 
relates the average value of the output amplitude spectrum to the input 


data. The theoretical result is 
et Renae |}=  *caey?wele2ao} + ROL (196.11) 
while for estimates the approximation is: 
<iRenaey > ~ @*caey?a<€2ap> + EOL 


(19¢. 12) 
{n=0, 123. S505 CS ° 1)3 


The reader should again be reminded that all the equations are written in 


terms of only the nonnegative part of the double-sided amplitude spectrum. 


A 


X(nAf), Amplitude Spectral Density 


40.00 


32.00 


-00 


au 


16.00 


00 


8.00 
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AMPLITUDE SPECTRAL DENSITY FUNCTIGN 


M 
(F -1) 
nAf, Frequency 


Figure 19C-2. Estimated Amplitude Spectral 
Density for Gaussian Noise (M=256)- 
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Using the notation 
2 
A 
o = $ Ccae)?m<E7(K)> + OI" (19¢.13) 


it is possible to express the characteristics of the Rayleigh distribution 


compactly as: 


1. Mean: <{X(nae)|> ~ Bo, (19.14) 
2. Mean-Square: <[X(nat) |2> > 20° (19C.15) 
3. Variance: <(|x(nat) | ~<[X nae) >)?> = (2 - Do? , (190.16) 


For the example with M=256, Equation 19C.13 gives (of = 125.3). 

In this section we have shown that the amplitude spectrum estimates 
are Rayleigh distributed both in an ensemble sense and across frequency. 

The statistical properties of a DFI estimate of the amplitude spectrum 
of Gaussian noise can be used to develop a simple test for the detection 
of a periodic component in the spectrum both over frequency and in an 
ensemble sense. 

First, at an a priori selected frequency aAf, the amplitude estimates 
are Rayleigh distributed over an ensemble of spectral estimates. The 
probability that a single estimate, Als from one ensemble function will 


exceed a selected threshold x, is given by: 
P = Pc |x | sx_) = expl-X2/207] (19¢.17) 
° a ° AP EE SOs : 


This is analogous to the well-known "false alarm" probability for statisti- 
cal detection (Whalen, 1971, Ch. 8) when using a Rayleigh envelope function 
(Whalen, 1971, Ch. 4). The detection criterion for testing an ensemble of 


spectral estimates for a periodic component at aAf is developed by selecting 
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A 
a confidence limit using (19C.17) and (19C.13) and testing all X's against 


the computed threshold X,- 


Second, for a single sample record of the estimated amplitude spectrum 
the amplitude estimates are Rayleigh distributed over frequency. The 
A 
probability that any single estimate, |X(naf) |, will not exceed a selected 


threshold X, is given by: 
A 
P({X(naé) | SX,) = (1-P) (19¢.18) 


This is simply one minus the false alarm probability. Using the theorem 


of joint probability and the multiplication theorem (Beckmann, 1967, p. 19) 


leads to the probability that any of the M/2 independent amplitude estimates 
will not exceed a threshold X,: 


A A A = _ M/2 
P(,; Xo» <a Xy/2 X,) = (l- Py (19¢.19) 


Likewise the probability that one or more amplitudes in a single spectral 


estimate will exceed X, is given by: 
nee 6s Ie ae 19¢.20 
o- Te) (19C€.20) 


sdustied 19C.20 can be used to test the largest spectral component to 
see how likely it would be to achieve that given level. If max{X(nAf) } ex- 
ceeds some chosen confidence threshold xy? we can use this test to determine 
the likelihood of a periodic component. As an example, for the record 


M=256, the largest amplitude is: 


max{ IX(nae) |}= IX(10a£) | = 37.4 (19C.21) 
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Computing o using (19C.13), the test probability using (19C.17) is: 
2 -3 
Pa exp[-(37.4)“/2(125.3) ]=3.77 x10 (19¢.22) 


The probability that at least one of the amplitudes would be as large as 


A 
X(10Af) is: 
A 
P=1- (1-2)? = 0.38 (19¢.23) 


This is a very likely occurrence so we can assume that the spectrum repre- 
sented by Figure 19C-2 is due exclusively to random noise. 
The threshold for a confidente limit of 0.01 is obtained by solving 


Equations 19C.20 and 19C.17 for X,° The solution is as follows: 


P=0.01=1-(1-P,) 7 


P =7.85x107> 
°o 


67 = 125.3 
fe] 


X_ =48.7 
) 


-If any of the estimated amplitudes had exceeded this level it would have 
been strong evidence for a periodic component in the data. This technique 
is recommended as a method of detecting a periodic component in noisy data 
by using the DFT. 

A second example will now be given that includes both sinewave detec- 
tion and the effect of increasing the total qunber of phase coherent samples 


(as discussed in the next section). A test signal of the form 


y(kat) = §(kAt) +A sin (2nfkAt + 0.77) (19C .24) 
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was generated using; Gaussian noise with zero mean and unity variance, 
A=0.707, and £=0.249. PROGRAM 05 was used to obtain amplitude spectrum 
estimates for data records of various lengths. The results and conclusions 
will now be discussed. 

Table 19C-1 is a summary of results for various data samples. The 
Rayleigh mean and standard deviation are shown along with the confidence 
thresholds computed for the 0.05, 0.01, and 0.001 confidence limits. The 
largest amplitude, its frequency, and probability of occurrence are also 
shown. Figure 19C-3 shows plots of the amplitude spectral density function 
for the various phase-coherent sample lengths. The effect of the sinewave 
component "growing" out of the noise is very evident. 

The sinewave component is easily detected for M=256 as shown in 
Figure 19C-3(d). The probability of occurrence of a component this large 
from noise alone is only 1.5x107>. This corresponds to a DFT spectral 
signal-to-noise ratio (Chapter XIX, Section E) of 31.9 or +15.0 dB (Equa- 
tion 19E.4), These numbers agree quite well with envelope detection theory 
(Whalen, 1971, Chapter 8). From this we conclude that enough samples should 
be taken so that SNR, (Equation 19E.4) is greater than 12-15 dB. 

This last example was intended to illustrate the effect of increased 
sampling on the detection of a sinewave component in noise and to indicate 


the number of samples necessary for a good probability of detection. 
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D. DFT and Spectral Averaging 
In Chapter XII the sampling and estimation problem was discussed and 

some groundwork was done on spectral averaging for noisy periodic signals. 
In Section C of the present chapter, the DFT amplitude spectrum estimate 
was characterized for a bandlimited Gaussian noise sample. In this section 
we will look at simple methods of spectral averaging and how they affect 
the estimate of the amplitude spectral density function. 

- For a Gaussian noise input, the amplitude estimate given by Equation 
19C.7 has an average value given by Equation 19C.14. For large samples 


the mean converges to: 
E{[2qnae) |} = acEMEte? ce) })? (19.1) 


The mean value depends on the sampling interval, the total number of samples, 
and the variance of the input noise. 

Now consider a finite data function {x(kAt): k= 0, +1, +2, ..., +N} 
represented by Equation 19B.1 and a rectangular data window function of 


width, Ty = (2N+1) At: 


. 27m 
+M ceo kAt 
x(kAt) = ECkAt) + = ce (19D .2) 
m=-M 


The amplitude spectral density is estimated using the sum of the DFT of 


§(kAt) and the spectral mixing formula (Equation 11.19): 


20 


‘ +N -j nk 
X(mAf)=At LC E(kdtje 
=-N 


+1 


Ps 


+M 

A 27m 

+ wy = CoG - nAf) (19D.3) 
m=-M F 
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A 
If we represent the second term of (19D.3) by the estimate, A(nAf), of the 


"true" spectrum and the first term by Equations 19C.3 and 19C.4 the result 


ne 
A A A A 
X(nAf) = t(a, - i8)) +A(nA£) (19D.4) 


A 
Since the estimate A is complex it can be represented as 
A “30. 
A(nAf) =Ale (19D. 5) 
and the estimated amplitude spectrum can be expressed in complex form as: 


X(naé) = Cath 8 j-jfac8 +A sine ] 6 
X(nAf) = Ata +A, cos le) Ate. oun e (19D. 6) 


The terms A, and a are really the noiseless estimates of |x(naé) | and 
arg{X(nAf) } discussed in Chapter XII. These estimates are corrupted only 
by the spectral mixing effect. The terms a, and a are Gaussian random 
variables which represent the effects of noise. To adequately represent 
the effects of record averaging, the terms in Equation 19D.6 need a sub- 
seript denoting the record number. If the record number is denoted by the 


subscript m then the following double subscript notation is needed: 
X (nat) =[at o a J- jfat $ ins] 19D.7} 
K (nd )=LAt Om? Anm ©°S beau es At Bam? Anam Si® Le (19D.7) 


The vector or coherent average for R records is expressed as: 


ee Te ae lea e | 
ve X (nd )}=C[at R Wate am £°S Pan 
n= m=1 
ie i. = 
~jfar 3 = tate EA sine] (19D.8) 
n= m=1 
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For large numbers of records the noise averages out and the result may be 
written in abbreviated notation as: 
A 
Ave{X(nAf) } =Ave{A cos 83 - j Ave fa. sin a3 (19D.9) 


The algebraic or incoherent average is obtained by averaging the mag- 


A 
mitude and phase separately. The magnitude and phase of X(nAf) are: 


IX (maf) | = fat + cae) (a2 + B) +2At Ae, cos @ + 8, sin® ) 7 


(19D.10) 
A 
4 -(At B tA, sin oe 
Arg{X(ndf) } = Arctan = ————— | (19D.11) 
(At @ +A _ cos® ) 


Since only the average magnitude is desired for an amplitude spectral den- 
sity estimate the two types of averages give the final results: 


1. Vector or coherent average. 

Ave{ IX(nae) |= [ave ts, cos ey + Avefa, sin ey 7 (19D.12) 
2. Algebraic or incoherent average. 

Ave{ IX(mae) |}= Lave (02) + cat)? ave(a- + 23 (19D. 13) 


The quality of the estimate of the amplitude spectrum is affected both 
by the cbservation period ty (or the total number of samples for fixed At) 
and by the number of records averaged. Equations 19D.12 and 19D.13 are the 
final results assuming that enough records have been averaged that the 
variability of the estimated average can be neglected. 

According to the spectral mixing theory of Chapter XI, as the observa- 


A 
tion period Ty increases, the estimate A converges to the "true" amplitude: 
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-58 
Ale n = X(nAf) = i (19D.14) 


and the coherent average gives: 


Ave{ IX(nas)]} = |x (nas) | (19D.15) 


£ the records are not phase connected, the phase angles Ci become random 


H 


functions of the record number with a uniform distribution (-1r,+m) and the 
coherent average becomes zero. Normally the phase connection will be ap- 
proximately correct and a will have a Gaussian distribution around the 
mean angle. If phase connecting cannot be guaranteed the coherent average 
should not be used. The coherent average is a fickle performer as an es~ 
timator and must be used with caution. 

To assess the performance of the incoherent average we need to express 
it in an alternate form by using Equations 19C.7 and 19C.14 to give: 

ave(a? + 62} = ae Dor (19D. 16) 


This is used in 19D.13 to obtain the alternate representation: 
Ave{ |X(nat) | }= [ave(a>}+ De} (19D.17) 


From (19D.3) we see that ae increases as i while from (19C.13) o increases 
only as Ty (M=T,/at). Since AL is always positive, the estimate for large 


Ty converges to: 
A 
Ave{ |X(naf) |} = Ave{a } (19D.18) 


For Ave{a_3=0, the estimate of (19D.17) converges to that of (19C.14). 
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The amplitudes Alm are statistically variable because the samples are 
not phase connected and because of spectral mixing effects. Because the 
phase reference for several arbitrary records is random and approximately 
uniformly distributed, Aum will have a Gaussian distribution about some 


average value that has a fixed bias from the true value, [x(na) |. 


E. DFT Signal-To-Noise Ratio Concepts 
The concept of a spectral signal-to-noise ratio seems like a natural 
extension of the presentation in Section D and may contribute to a Bice e 
understanding of the spectral averaging concept. If we define a signal-to 


noise ratio for a Fourier frequency nAf as 


ale |? : 
SNR - oo (19E.1) 
mS <7 (q)> } 
and use the relations, 
2.1 2 
a, =D At T, <8" (kp? (19E.2) 
lc | == [Xcnasy | (19E. 3) 
n Ty ‘ 
the SNR at nAf becomes (M=T/at): 
Ronen? 
SNR = 2 (19E.4) 
—a 


The average value of the estimated amplitude A, can be represented as a 


correction factor y, times the "true" amplitude: 


A 
AvefA }= ve IX(maé) | (1SE.5) 
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If we identify the total number of samples as M=T,/at then Equation 


19D.17 may be written as: 


i 


A 
ave{ |X(na£) | }= 7c, 1+ (ye SNR 7 (19E.6) 


This equation expresses the average amplitude spectral density as a func- 
tion of the signal-to-noise power ratio for a spectral component repre- 
sented by the complex Fourier coefficient, c. 

For a sufficiently long sample of a model consisting of Gaussian noise 
with a single periodic component represented by Cc. the spectral mixing 
correction factor, Ya? is unity for n=m and zero for all other n. This 


leads to the approximations: 


avef [xcnae) |} = @%o, atm (195.7) 
avef [X(ma£) [3 > (75,t1+ (2) snr 7 | (19E.8) 


For a single sinewave we will briefly introduce another concept with the 


help fo Equation 19E.8. Applying Parceval's theorem (Equation 18C.5) we 


get 
+N, A +N i | ‘ 
At 5 x(k) = soe SCOTS Ave { [X(nat) |} (19E.9) 
k=-N (W+ Dat y 


or after many manipulations: 


Tm 2 


J ofi++ sme ] (19E. 10) 


Mat)? <x” (k)> = 


The presence of a sinewave increases the mean-square value of x(k) as in- 
dicated by Equation 19E.10. If the removal of a spectral component mAf . 


decreases <x? (k)> by an amount that is statistically more significant than 
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the removal of a random component, then (19E.10) could be the basis of a 
criterion for detecting a single sinewave component in noisy data. 
The author plans to publish additional work with these topics in a 


future paper. 


F. MEM in the Z-Transform Domain 
In Chapter XV it was shown that the maximum entropy spectral estimator 


could be represented by (Equations 15D.38 and 15D.39): 


, ae ATP. - ATP, 
is ee 2 (19F.1) 
Q(z)q'(z) ~—facz) | 
where: 
z= etientar le| <tr ~ (19F.2) 
= ~k 
Q(Z)=le 2 ay (k)z (19F.3) 
k=1 


Remember that AT is the sampling interval in the autocorrelation domain, 
that a, (k) represents the coefficients for a K-th order autoregressive 
model of the time series, and that Pe is the minimum mean-square error of 
the one-step prediction. The properties of the spectral estimator can be 
related to the properties of the complex polynomial Q(Z) and the aeeociated 
coefficients, ay (k). 

The orders of the polynomial and the process are always the same. Q(Z) 
also represents the z-transform of the "best" least-squares linear predic- 
tion filter for the process. For this reason lacz) {2 will be called the 
absolute value squared of the filter transfer function or simply the power 


transfer function. We will use the abbreviated notation 
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6(£) = |a¢z) |? = JaceTs27*475 |? (19F .4) 


to indicate that Z is evaluated on the unit circle so that G(f) is real. 
The notation is not precise but generally there is no confusion as to 
peeatne Another notation option will be the substitution of at= 1/2f, 
into some of the equations so that the effect of the Nyquist frequency 


limit can be displayed directly: 


Z=e je| s £. (19F .5) 


As a Simple example we will look at a first order process with associ- 


ated polynomial and power transfer function: 


Q(z) =1-a,(1zt i (19F .6) 
laczy |? = C1 -a,(27 Ifa - a, (zt 
= -aZ + ee a (1) j< a,z* (19F.7) 


The zero of Q(Z) occurs at 2, = a,(1) and those of laczy |? at zy 9=a,Q), 
> 


L/a, (1). Figure 19F-1 shows the estimated power spectral density for dif- 


ferent values of a, (1). 


Now consider a second order process: 
-1 <2 
Q(Z) =1- a, (1)Z - a, (2)z (19F.9) 


Jacz) |? = -2,(2)27* ~ a (1) [1 - a,(2) 27? 


+ [1+ af (1) + a,(2) J~a,(1) [1 = a,(2) 12! - a,(2)27 (19F 10) 
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Figure 19F-1. First-Order Spectra 


G(£) = [1+ a(1) E a (2) ]- 2a,(2) cos (2né/£,) 


- 2a, (1) {1- a,(2)]cos (£/£,) (19F.11) 
The zeros of Q(Z) now occur at: 


oO) 4% [abet +4a,(2) 77 (198.12) 


" 
tole 
ry) 


An excellent example for a second order autoregressive time series is given 


by Fuller (1976, p. 55): 
x(t) - x(t - 1)+0.89 x(t - 2) = E(t) (19F. 13) 
a,(1) =1 a, (2) = -0.89 (19F.14) 


The zeros of Q(Z) now occur at: 


346 


Z1 > = 9-500 + j 0.800 


> 


= 0.943 et J 1-01 (19F 15) 


A plot of the locations of both zeros is shown in Figure 19F-2. The auto- 
correlation function for this process has the appearance of an exponentially 


damped sinusoid. The power transfer function is: 


G(f) =2.79+1.78 cos (2n£/£,) - 3.78 cos (m£/£.) (19F.16) 


+j Im{z} 


Z-Plane 


Re{z} 


1 2 


Figure 19F-2. Zeros of Q(Z)=1-Z +0.89Z 
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200 


100 


0 +0.5 fy +h, 


Figure 19Fs3. Typical Spectral Peak for MEM 


1 1 : : i. 
A plot of anBy S,() = GE ‘ts shown in Figure 19F-3. A multitude of the 
properties of MEM spectral estimators can now be summarized using the second 
order process as a typical example. These properties are: 

1. Q(Z) gives a complete description of the spectral estimate and the 


location of its zeros carry important information about the spec- 


tral shape. 


7. 
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Q(Z) is defined so that all of its zeros lie inside the unit 


circle. 

For second and higher orders, the zeros of Q(Z) may be complex. 
Zeros lying close to the unit circle are capable of representing 
nearly periodic structure and thus allow MEM to estimate both 
noise and quasi-periodic components. Zeros off the real axis al- 
ways come in complex conjugate pairs. 

For each quasi-periodic component, two orders are needed for Q(Z) 
because the zeros must come in complex conjugate pairs. Thus to 
describe five quasi-periodic components the polynomial must be of 
at least order ten. 

The Burg recursion restricts the K-th autoregressive component to 
the range: 0 <a (K) <1 thereby assuring a positive definite co- 
variance matrix and the absence of zeros lying exactly on the unit 
circle. 


The frequency of a zero that is sufficiently isolated from other 


zeros can be obtained approximately by: 


fn Im{z } 
f, ~~ arctan ete} (19F .17) 


For the Fuller example, £, = 0.322 f.. 

Some estimates can give a very sharp peak as illustrated by Figure 
19F-3. When designing a numerical algorithm for the estimator one 
must guarantee that no spectral peaks are missing. A quality 
check can be performed by integrating S_.(£) to see if it gives the 


total power. 
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When the approximate peak locations are known an expanded resolution as 
illustrated by Figure 19F-4 may be economical. 
Another approach to determining the location of spectral peaks is to 


determine the zeros of Q(Z) directly from a polynomial solving routine: 
» 4 7 a = K 
Q(Z) =(Z Z,)(Z Z,)(Z Z3) sae Z)/2 (19F.18) 


This works especially well if the zeros are well separated and the order is 
within reasonable limits. This approach also has the advantage that the 
stability of the zero locations and their separation from the unit circle 
can be more directiy observed as K is increased. An example of this ap- 


proach will be given in Section H of this chapter. 


200 


100 


$e tf £ 
+0.2 fi +0.3 fy +0.4 fy 


Figure 19F-4. Expanded Resolution of Spectral Peak 
Showing the General Shape to be Expected 
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G. MEM Resolution and Stability 
This section is essentially an extension of the last section with an 
emphasis on the resolution and stability properties of the MEM spectral 
estimate. The estimator given by Equations 15F.1 or 19F.1 is Show 66 be 
continuous, however, for a practical computer implementation, it is dis- 
crete. The practical display resolution, Af, is limited by the amount of 
computational effort one is willing to invest into Equation 15F.1. A dis- 


crete estimator will be represented by 


ATP. 


§ (maf) 2 = Ko (196.1) 
k=1 
where: 
{n=0, 41,42, ...,+(£,/A£)} (196.2) 


The analogy between MEM and digital filtering will be fully exploited 


in this section. To begin we redefine the complex polynomial to agree with 


H(Z) = w =D (196. 3) 
where: 
D(Z) = 4, (0) +d, (1)Z +4, (2)27 + 22. +d, (RZ (196.4) 
| ~D(Z) = (Z~Z,)(Z-2,)(Z-Z,) «2. (Z- 2) (196.5) 
z= et i2ntr | (196. 6) 


The coefficients of D(Z) can be identified with those of Q(Z) by the 


following: 
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d, (RK) =1 d (K = k) = ~a, (k) (196.7) 


The complex transfer function H(Z) represents the moving-average filter 
model, The zeros of Q(Z) are also the zeros of D(Z). The K-th order zero 
of H(Z) at zero represents a phase angle of Knf/f, at £ but has no effect 


on the spectrum. The MEM power spectrum can also be determined by: 


A * : 
S(£) = ATP. H(Z)H (Z) (196.8) 


as et i2ntat 


It is common practice to represent each factor of Equation 19G.5 as an am- 
plitude and phase 
+j a 
(Z- Zz. =re (196.9) 
so that (19G.5) can be expressed in an alternate form as: 


+j(8, +6 +...+6,) 


1 1 °2 
D(Z) = Ty Tors °° re (19G.10) 


The magnitudes, tT, represent the magnitudes of a vector from the zero 
location to a location on the unit circle and o is the angle this vector 
makes with the abscissa. The filter transfer function can now be written as: 
+7 =. > eco ™ 
j(Rnt/£, 8, "8, 0.) 


H(Z) = £ (196.11) 
TyTok3 eee % 


The spectral estimate becomes: 


5 ee (19G.12) 
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The concept of a complex frequency location to describe the zeros of D(Z) 
will now be given. Its use will become apparent later. A complex frequency 


om is defined as 
Qo = is j a (196.13) 


and is used to describe the location of a zero of D(Z) in complex notation: 


+j7Q_/ 
e aN (196.14) 


nr fy +3 fy 


Zs 
n 


Z =e (196.15) 


n 


The factors in (19G.9) are expressed in terms of a complex frequency as: 


+j2n0/f, 
) 


(2-2) =(Z-e (19G.16) 
On the unit circle we get 

(2-2) = teow - es nen! fn (19G.17) 
or alternately: . 

oe Rau rr : Smeg yt Bue ake er 


The absolute magnitude of (19G.18) is: 


-2na/ fy om / fy 
- 2e 


[z-z =r = {ite cos (n(B, = £)/£,) (19G.19) 


The spectral shape of Equation 19G.12 is dominated by the function 
jz S2.\° near an isolated zero, Za that is close to the unit circle. The 
function x is conceptually very important because it represents the generic 


shape factor of the MEM spectral estimator near all isolated zeros of D(Z) 
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when the zero is near the unit circle (small a: = is a function of both 
parameters, a and ey and of the frequency variable f. This is the 
reason for the concept of a complex frequency to represent the zero. In 
general we can use (19G.12) to represent the spectral estimate with the 


amplitude terms 2 formally defined for each zero Z_ as: 
n n 


-2na_/ -Ta / 
r(f)=[1+e = MWesge ‘ Wey (m(B, - £)/£,)] (19G.20) 
with: 
fy . 2 
eames tn{|z_ 13 a >0 (19G.21) 
fn 
B= Arsiz} -£, $8 S+fy (196.22) 


Consider the example shown in Figure 19G-1 for a 3rd-order process 
with a zero on the real axis and a single complex conjugate pair. For fre- 
quencies very near the zero, Z3> we have n£/f, = 6, and f ™ 8B. The esti- 


mated spectral density is approximately: 


ATP 
K 
§ (£) = 


-2na,/ -7a.,/ 
r7(8,)25( 83) {l+e : Wo 2e 3 N cos (mB, - £)/£,)] 


(19G. 23) 


For a<h, the spectral peak is influenced only by the closest zero 


and occurs at f= B,- The estimated spectral density has a peak of: 
A ATP. 


K 
Se ae ee ee (196.24) 
ri(B,)x5(B [ite > Nog 3B 
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For a < fy the peak is symmetrical about f= a If we define the 


‘ : 
frequencies Bt Af/2 as the points where S(£) is down from the peak value 


by an amount XK. then the bandwidth Af can be determined from the expression: 


+jIm{z} 


frequency = £ 


Zeros of D(Z) 


Figure 19G-1. Complex Plane Interpretation of an MEM 
Spectral Estimate for a 3rd-Order Process 


-2ne [ty oT [fy 
n n 
(ive = oe ; -2e s = (196.25) 
(8, + Af/2) ° 


The solution for Af in terms of the cosine function is given by: 


cos (TAS) = K[1-KKj 1 (196.26) 
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-To/ ono_/ -2re_/ 
x, - Lite at 20 alt K ie 196.27 
1” -2T1@ /f, 2° 2 =n /t, (196.27) 
{l+e os } 2e ™ 
af _2 Z : 
== ARCCOS {K, [1 Kk, I} . (19G.28) 


N 


For smali a? Ky is very close to unity. As an example, take the solution 
represented by Equation 19F.15 and Figures 19F-3 and 19F-4. The appropri-- 


ate factors are: 


£ 
# =x tn{0.943} = - 3 (-0.0587) (196.29) 


R 
u 


= -3 ~ 1-¢ 
K, = 1.72 x 10 K, = 1.0017 


Assume a width where the spectrum is down by a factor of K,=2 and the 
solution is: 


sf, 2 
= ARCCOS {0.9983 } = 0.0374 (196.30) 


This result can be verified from Figure 19F-4. 
Next we consider the spectral resolving ability of MEM for two closely 


spaced zeros near the unit circle. Assume they are symmetrically spaced 


about a nominal frequency By such that: 


B.=B +Aaf/2 
i to (196.31) 


By = 8, - Af/2 


A 
Peaks occur in S_(£) at f= 8)> 8, and a valley occurs at f= Bo 
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First assume the zeros are equidistant from the unit circle so a, = 


Wy = a Then the ratio of the dip to one of the peaks can be written as: 


2 
r(B)r5(8,) = 
ee = K 


(196.32) 
2 2 fe) 
x5 (B,)F5(8;) 


Equation 19G.32 can be evaluated using the results given by (19G.19) and 


the frequencies, 8 =f and 8,=8 +Af/2. Expanding the terms gives: 
q fe) 1 fo) 


~2na,/ i ~re,/ fy 


fit+e - 2e cos Ge ake 
een a 
~2T, / -T, / -2ra_/ ono _/£ 
[1+e Noy Nae ore or Ne cos (OE) 
(196.33) 
This results in a quadratic equation in cos (e) given by 
[1+ 2K, K Jeos” ¢ Ge) - 2K, cos Ge 
2 
+ [Ke -K K(k, +1)]= 0 : (19G. 34) 
where: 
-2ra_/ -ttay_/ -2ra_/ 
{l+e oP Hg ? ) l+e oN 
K, = and K, = (19G.35) 


1 =r, / fy | 2 ~T,/ f. 


For two zeros spaced from the unit circle such that oF O01 ({z| = 0.99) 
and for a peak-to-valley ratio K,=2 the quadratic equation can be solved 


to give: 


bf ~ 9.0556 (196. 36) 


‘x 
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The solution of (19G.34) must be done with high numerical precision because 
the coefficients are so close to a value corresponding to Af=0. 

These equations illustrate the type of resolution, Af, needed to evalu- 
ate Equation 19G.1 when the zero locations are known from evaluating the 
polynomials D(Z) or Q(Z). A check on the accuracy of g (£) is to numerically 
integrate it to be sure it is equal to the total power. If any peaks are 
unresolved the integration will yield a value less than the total power. 

A solution for resolving power when the zeros are not equidistant from 
the unit circle is much more difficult. Simplifying assumptions about the 
location of peaks and valleys are no longer valid. An estimator for the 
locations of the peaks and valleys as a function of Af would have to be de- 
termined. This more general problem has not been solved. 

To determine stability and accuracy of the estimate we need to relate 
the data to the computation of the zeros of D(Z). This will be done for a 


second-order process. For a second-order process the polynomial is 
D(Z) =-a,(2) - a,(1)2+2" (196.37) 


and the zero locations are: 


1 


Lom Fag) + $ [af (1) + 4a, (2) 2 | (196.38) 


Z 
If the roots are complex (a, (2) <0 and |4a,(2) | >ab(1)) the solutions are: 


21427 7a) +5 it4la,(2]- ay F (196.39) 


Examples of second-order processes can be found in many texts. A good ref- 


erence is Fuller (1976, p. 54). 
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For complex roots the following solutions are obtained for the root 
locations and the sampled autocorrelation function of the process. The 


absolute values of the roots are: 
Iz, 1 = Iz, | = la, ¢2) |? (196.40) 


The sampled autocorrelation function for positive k is (Fuller, 1976): 


R (kat) = A(KAT) sin rs kat +] (19G.41) 

cose) = $ a(D fa, (2) [2 (196.42) 
l-a,(2) mB 

Tan(¢) = T¥a,@) tan Sa, (196.43) 
[a (2) k/ : e72TOKAT 

A(kaAtT) = rr oa aoa ae (19G.44) 

From the concept of complex frequency we identify: 
a=- a tn{|a, (2) [* (196.45) 
B= *x arccos { a, (1) |a,(2)|7} (196.46) 


Keeping these results in mind we can now consider the Burg algorithm 
for MEM and possible sampling effects. First of all we know that a, (K) is 
determined by minimizing the error of M (Equation 15G.6) in a least squares 
sense. We also know that the total amount of data used for the estimate 
(Equation 15G.8) decreases with increasing K. This means that the variance 


of a, (R) for a given set of data increases with increasing K. Alternately 
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for a fixed value of K (K=2 in our example) the variance decreases with 
increasing amounts of data. Also as the data increases the variance of the 
estimated error Pe (Equation 15G.4) decreases. The general trends of these 
variances are shown in Figure 19G-2. The concept of variance for a, (K) can 
be shown using Equation (15G.12) and defining an ensemble with index N to 


illustrate dependence on sample length: 
var {a, (KN) }=E{ag(K,N) }20 N<@ (196.47) 


For large amounts of data the expected values of Pe and a, (K) tend to 
zero for large K. For smaller data sets they tend to decrease until the 


mean is masked by the variability. For some maximum K=K: 


Ela, &)}=0 (196.48) 


Eie, } = Py _jll-E*fa, CO pap eer (196.49) 
m m m ; m 


Typical trends are shown in Figures 19G-3 and 19G-4. For actual data 
records the masking effect is shown in 19G=5. 

The variability of |a, (x) | directly affects the distance between a 
zero and the unit circle (Equation 19G.45) thus affecting the peak value. 
In practical applications this can be seen as a change in peak value for 
increasing K and as the data record changes. 

MEM is becoming more widely known and used because it is an estimator 
which has better spectral resolution than the DFT for short data records. 
This effect can be explained for certain types of common sampling situations. 


The two most common examples are: severe oversampling of short data records 
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N= Fixed Number of 
Samples 


var{a,(K,N) } 


Increasing K eS aoe K=N 


K= Fixed Order 


var{a,(K,N) } 


Increasing Ne 


Figure 19G-2. General Trends of the Variability of 
(K) for Fixed Data Records and for 
ariable Data Records 
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Figure 19G-4. Typical Trend in E{P 3 for Large Amounts of Data 
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Fixed N 


Increasing K “~~ N 


Figure 19G-5. Typical Trend for Actual Estimates of the 
Total Mean-Square Error of the One-Step 
Prediction 


with one periodic component and a reasonable amount of bandlimited noise or 
long observation periods with a moderate sampling rate and bandiimited 
noise. 

Since the autoregressive coefficients can implement both a one-step 
predictor in time (Equation 13.35) as well as autocorrelation (Equation 
14.42) and since the coefficients are determined by minimizing the mean- 
Square error (Equation 15G.6) and are not explicitly dependent on the 
sampling interval (At=AT), the estimate can be considerably improved by 
oversampling. Not only does it decrease the variance of a, {K) due to noise 
but once the data record is long enough to include more than about 60% of 


a cycle of the sinusoid represented by Equation 19G.41 the extrapolation 
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property takes over and produces a strong periodic component. The extra- 
polation is helped by oversampling. For longer records the sampling prop- 
erties are analogous to those discussed in Chapter XII Section D, Part 4. 
In effect a single period of R(t) is oversampled and the extrapolation 
property completes the task of producing a sharp peak. 

The effect of increasing the order of the estimate on the spectral 
resolution is less obvious. For a given fixed record length, increasing 
the order increases the number of zeros of D(Z). Now only zeros close to 
the unit circle produce a significant effect on the spectral shape so for 
given periodicities in the data only these zeros are needed. For a finite 
and noisy record however these zeros are required to represent a least- 
squares estimate for all of the data. The net effect is to shift the 
zeros away from the unit circle. By increasing the order, K, of the esti- 
mate, additional zeros are added (additional degrees of freedom for the 
prediction) that are far from the unit circle thereby having little effect 
on shape and at the same time allowing the important zeros to move closer 
to the unit circle. For this reason the spectrai resolution is improved 
(sharper peaks) and the location of the peak is more accurate. The resolu- 
tion of two closely spaced and equal peaks is a function only of the dis- 
tance between their zeros and the unit circle. Figure 19G-6 illustrates a 
spectral representation of the effect of zero location. 

A particular zero, Z? has a random distribution in a circular sense 
around some nominal location. The distribution for Iz, I is a cross section 
of this circular distribution. When Z is close to the unit circle a small 


change in location has a very gross effect on the amplitude and location of 
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the peak. This shifting and variability can be seen for any practical 


record. A point of diminishing returns is reached where the average zero 
location is no longer moved closer to the unit circle by increasing the 
A pictorial 


order, K. The order should not be increased beyond this point. 


representation of this effect for an isolated zero near the unit circle is 


shown in Figure 19G-7. 


To Origin 


0.9 


[Zn| 


Unit Circle 
1.0 Ft 
A £ 


re{z,} To==T 


‘Ny 


Figure 19G=-7. Variability of the Estimated Zero Location 
; for Increasing Orders of the Estimate 
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H. DFT and MEM Resolution Limits 

1. Introduction 

When considering the resolution of a spectral estimator, one must be 
careful to distinguish between the ability to detect and resolve periodic 
components and the frequency resolution to which the resulting weeiuets is 
displayed. The latter will be referred to as the display resolution while 
the former will be called the spectral resolution or simply the resolution 
of the estimator. 
| An increase in display resolution is only a matter of increasing 
processor time. It makes the estimate appear more continuous and may be 
used to produce a more pleasing appearance. High display resolution is 
particularly needed for MEM because a sharp peak may otherwise be over- 
looked. For the FFT, higher display resolution may sometimes improve the 
frequency ambiguity of a spectral peak, 

In contrast, the spectral resolution of the estimator is an inherent 
aeeperey that is determined by the "quality" of the estimator and the ob- 
servation time for the data. . It cannot be improved by postprocessing 


techniques. 


2. Display resolution 


For MEM spectral analysis the information about sharp spectral peaks 
is all contained in the zero locations of the complex polynomial, D(Z). 
These sharp pedis will always be displayed if the frequency interval Af 
(Equation 19G.1) is kept small enough. For a given polynomial, the dis- 
play resolution can be arbitrarily increased by decreasing Af without af- 


fecting estimator quality. 
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For the FFI, the display resolution problem caused by a saingied spec- 
trum can be minimized by either using a continuous estimator (Equation 
11.1) or by concatenation of zeros to the record. Zero concatenation is a 
well-established technique in FFT analysis and is often used to make the 
number of data points the required power-of-two. Concatenation forces the 
FFI to treat the data as though it were a periodic time pulse with a low 
duty cycle. For large orders of concatenation, the spectral estimate 
closely approximates that for a single time pulse. Concatenation also in- 
creases the apparent observation time thereby causing the apparent ampli- 
tudes and frequencies to shift slightly. This can sometimes result in a 
slightly better estimate of the frequency of peak amplitude. The disad- 
vantage to large order concatenation is increased processing time. A 


simple DFT does not require zero concatenation. 


3. Spectral resolution 


The fundamental resolution limit for the DFT is imposed by Ty and was 
given for two closely spaced sinusoids by Equation 12B.48. The Rayleigh - 


resolution or one-half the bandwidth-between-first-nulls is given by: 


(19H.1) 


This resolution limit assumed a continuous estimator (Equation 11.1) and 
peaks of equal amplitude. Neither assumption is the case in most practical 
FFI applications. In most FFI analysis, the estimated location-of-peak is 


1 , F ; 
within + =~ Hz. of the true peak. Zero concatenation does not improve this 


Ty 


resolution. 
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Improvements in the spectral resolution of the DFT can only be ob- 
tained by increasing the observation time, Ty 

The spectral resolution for the MEM spectral estimator is limited by 
the number of independent samples of the time series (observation time, Ty) 
and the degrees-of-freedom for the spectral estimator (order of the esti- 
mated autoregression). Increasing the number of samples decreases the 
variance of the estimate of the expected values of the autoregressive co- 
efficients. Increasing the order of the estimate allows more degrees-of- 
freedom to approximate higher order regressions thereby minimizing the 
error associated with the most prominent spectral peaks. This choice of 
the degrees-of=freedom is contrasted with the situation for the DFT where 
the order is fixed. 

Mathematical derivations for MEM resolution as a function of the 


number of samples and the order of the estimate have yet to be published. 


4&. MEM resolution limits 

The resolution limit for MEM spectral analysis in terms of the number 
of samples and the order of the estimate is much more difficult to assess 
than for the DFT. For the latter, the degrees-of-freedon are fixed due to 
the periodic assumption and hence for each two time samples there is an 
additional pair of orthogonal components. Also, there is no least-mean- 
Square error to be minimized. In MEM the degrees-of-freedom are selected 
by the user and hence the resolution limit is affected both by the number 
of samples and the order of the estimate. 

The presentation of MEM resolution in Section G of this chapter ad- 


dresses the properties associated with the location of the zeros of D(Z) 
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but does not present the effects of the number of samples or the order. 

Some of the present MEM literature refers to the assumption that MEM uses 
all of the data at each step in the recursive algorithm. It does not! End — 
effects, although minimized in the Burg algorithm (Equations in Chapter XV, 
Section G), are not eliminated for higher order autocorrelation components. 
Reduction in the amount of data used is clearly evident in the summation 
index of Equation 15G.12. End effects cause an increase in the variance of 
the autoregressive coefficients as the order is increased (Figure 19G-2) 
because of the reduced amounts of data used in the estimate. 

Parzen (1969) has suggested, but did not prove, a method for deter- 
mining che osdee based on the variance of the least-mean-square error esti- 
mate. Several examples examined by the author show that this technique 
produces an eStimate that does not have the best resolution inherent in 
the estimator. A better criteria is the stability (or variance) of the 
location of a critical zero of D(z). 

A general rule is to choose a sample length that will span more than 
one-half of the period of the lowest frequency component and chose an order 
high enough so that the variance of the zero locations of spectral peaks 


of interest is minimized. A detailed example will be given at the end of 


this section. 


5. DFT resolution limit 

From an estimation theory viewpoint, the resolution limit for the DFT 
is given by Equation 19H.1. It states simply that the resolution is limited 
by.the observation time, Ty From a theoretical viewpoint, the exact effect 


of DFT sampling on the estimated amplitude spectrum of a periodic function 
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is specified by the estimated Kronecker delta function defined in Equation 
12D.11 and used in Equation 12D.10. | 

Figure 11-1 illustrates the bandpass or weighting effect represented 
by beg Net » T,) for a single spectral component. The weighting function 
is specified for each harmonic number or frequency. The true amplitude of 
the spectral component is slightly reduced with the corresponding power 
distributed to other frequencies. The resolution limit for smaller ampli- 
tudes is limited by the peak value of spurious components. The magnitudes 
of the spurious components can be related to the "parent" peak by using 
the spectral mixing formula. 

For a large peak at frequency fi? the principal and spurious complex 


amplitudes can be obtained from a single term of the series in Equation 


11.20: 
eet eee ee ae 7 cos(2n£. - 2™)x nN 1} (198.2) 
is LNT 1 a L~ 1) * +i . 


For estimation purposes, c is replaced by its estimate when calculating 
Spurious components. An example of this effect is illustrated in Figure 
12D-3. For Ty =3.1, T= 1.0, and (2N+1)=21, the principal component and 


frequency are: 
lc, |=2.0 £, = 0.968 Hz 
The nearest spurious component has an amplitude and frequency of: 


ce. ,,| = 0.186 =1.29 Hz 
ley 


hel 
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A small adjacent spectral component could not be resolved if its amplitude 
did not exceed lesa 

The location of a spectral peak near zero frequency can be masked by 
the average value if only a fraction of a cycle is sampled. This type of 
error is illustrated in Figure 12D-2. Since partial cycles cannot be re- 
solved using the DFT it is highly desirable to remove the average value 


before processing. 


6. Example for MEM 


The interferometer data used as an example in Sections D, F, and G of 
Chapter XVII is also an excellent practical example for illustrating the 
effect of filter order on the MEM spectral estimator. The output from 
PROGRAM-07 in Appendix III provides the data used in this example. 

The time series (HER A 7/29/75) was processed using a variable filter 
length, K=1-15. The estimated autoregressive coefficients are printed 
for each order of the estimate. Figure i9H-1 illustrates the variability 
of ie estimated coefficients as a function of the chosen filter length for 
K=1-9. The variation of the magnitude of these estimates directly affects 
the zero locations of the z-transform polynomial they generate. The most 
variable coefficient is a, ({®) because it is the independent variable for 
the one-step prediction. 

The estimated coefficients were used to generate the complex poly- 
nomial, D(Z), as described in Equations 19G.4 and 19G.5 and the roots were 
obtained using a polynomial root solver. A summary of the results for 
filter orders up to ten is shown in Table 19H-1. Each root is described 


by a top entry which is the magnitude (lzI< and a bottom entry which is 


372 


the phase angle in radians (-n sargiz} S+m). These root locations are 
plotted on the unit circle as shown in Figure 19H-2. The zeros represent- 


ing the principal peak are labled Z, and are marked with a dotted line. 


lt 2 3. 4 5 6 7 8 9 Order, K 


Figure 19H-1. Variability of the Estimated Autoregressive 
Coefficients as a Function of the Order, K, 
of the Estimate 


The estimated location-of-spectral-peak for Z, varies as a function 
of the order K. Figure 19H-3 illustrates this variation. This character- 


istic was discussed in Section G and an example plot given in Figure 19G-7. 
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Figure 19H-3, Migration of the Estimated Location of 
the Spectral Peak Represented by Zy 


Several interesting features of the MEM spectral estimator are illus- 
trated in this example. The first thing we realize is that the mean-square 
estimation error is not a suitable measure for selecting the order of the 
estimate. From Figure 17G-5 we see that the error decreases very little 
beyond order K=2 and yet Figure 19H-3 shows the estimated location-of- 
spectral peak improving a to order K=15. This improvement comes about 
because the degrees-of-freedom were increased thereby allowing the zero 


location to migrate to its preferred position. Also the peak became 
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sharper as indicated by the magnitude of Z, approaching unity. The impor- 
tant conclusion is that the zero location is a better indicator of the 
desired order of the estimate. 

Figure 19H-2 shows the effect of increasing order on all zero loca- 
tions. It is important to note that each spectral peak to be resolved 
requires two additional oases of the polynomial. This is illustrated for 
order K=5 where two peaks are modeled. In other words, only those zeros 
in the upper-half-plane have a significant effect on spectral shape. From 
Table 19H-1 and Figure 19H-2 we also see that the identity of the most 
important zeros can be preserved over many orders of the estimate. This 
property also helps in the selection of a maximum order. 

The techniques of analysis presented in this example are typical of 
those that may be used for practical MEM spectral estimation. For each 
prominent spectral peak a separate plot similar to Figure 19H-3 should be 
made. The "true" location is marked for reference and illustration. In 
general an estimate of the true location must be made. Estimates of the 
total power in a spectrai peak must be obtained by computing the area under 
the peak. This power spectrum technique is in direct contrast to FFT anal- 


ysis where the amplitude of the peak is important. 


I. Review of Common Spectral Estimation Errors 
1. Introduction 
This review of some of the more common errors associated with the es- 
timation of amplitude and power spectra will specifically address the DFT 
(also FFI) and MEM. The techniques and effects of data preprocessing on 


errors have been discussed in Chapter IX, Section C. Here we concentrate 
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on the major effects of observer bias and error and intrinsic estimation 
errors, Gross observer error, equipment malfunction error and lack of 


Stationarity will not be addressed. 


2. Aliasing 


Aliasing or the frequency fold-over effect caused by sampling at a 
rate less than the Nyquist rate has been discussed in Chapters VI and IX. 
In the FFT literature, aliasing has been given considerable attention while 
in MEM literature it is seldom mentioned. The reader should be keenly 
aware that it can occur in both types of analyses. 

The effects of aliasing can be minimized or eliminated by careful ex- 
perimental design such as prefiltering ahead of the sampler. Under certain 
special circumstances aliasing is desirable. An example cf a pructical 
application of aliasing is for the detection of a sinewave in a spectrum 
mich wider than the principal alias. The fold-over effect allows the sinu- 
soid to be detected without an increase in the sampling rate. 

For the vast majority of spectral analysis applications aliasing must 
be eliminated and, when reporting results, it is helpful to inform the 


reader as to how this was accomplished. 


3. Numerical stability and accuracy 


Because the word length is finite for digital processing systems, it 
is possible to introduce noise and instabilities into the various spectral 
estimation algorithms. Truncation or roundoff error will introduce wide- 


-band noise into the estimate. 
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Truncation can be serious for MEM because, for good signal-to-noise 
ratios, it can cause a zero of the characteristic polynomial, D(Z), to lie 
on the unit circle. Some applications may require the use of double pre- 
cision or the addition of a small amount of wideband noise to eliminate 
this effect. A well designed MEM algorithm should check for this effect. 

For a majority of practical scientific data processing, numerical 
stability and accuracy will not be a problem. Only for the cases of very 


high predictability or very small word length should the user apply special 


precautions. 


4. Spectral smoothing 


Spectral smoothing or the loss of spectral resolution caused by a 
finite observation time is discussed in Chapter IX. Smoothing in FFT anal- 
ysis is caused by a finite ine eandoas In the Blackman-Tukey method it is 
caused by a finite lag window. Smoothing in MEM analysis is the result of 
using a smoothed estimate for extrapolating the autocorrelation function 
and using a single zero of the complex polynomial, D(Z), to represent a 
single spectral peak. 

The undesirable effect of spectral smoothing can only be reduced by 
increasing the number of samples (or the observation time). For MEM, im- 
proved spectral resolution (over FFT) is the result of using an estimator 


that has a selectable degrees-of-freedom. 
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5. Spectral leakage and end effects 


Spectral leakage (or spectral mixing) is caused by the finite obser- 
vation time (finite time window) and is discussed in Chapter IX. The spec- 
tral mixing formula derived in Chapter XI is an exact model for represent- 
ing the affect of a specific DFT estimate on spectral leakage. For MEM, 
spectral leakage is observed as a shift in the absolute magnitude of the 
zeros representing spectral peaks. Leakage reduces the ability to resolve 
minor components peeause a large peak may have leakage that obscures a 
minor one. For DFT analysis the spectral mixing formula can be used to 
place an upper bound on leakage and hence on minor component detection. 

For MEM, no technique, as yet, exists. 

The variability of estimated autocorrelations with increasing lags is 
caused by the end effect of finite data records. For the Blackman-Tukey 
method, end effects are minimized by the proper choice of a lag window 
(Blackman and Tukey, 1958). For MEM, end effects are minimized by the 
least-squares smoothing used in extrapolating the autocorrelation function. 


End effects for MEM are further discussed in Section G of Chapter XV. 


6. Noise and statistical variability 


Sample records of the time series representing a physical process are 
usually contaminated by random noise inherent in the measurement process. 
Also, since the sample records are finite, the estimated means of the 
various parameters have a considerable variability that depends on the 
length of the sample. These effects introduce random noise and statistical 


variability into the various spectral estimators. 
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The effect of noise on the DFT estimate was discussed in Section D of 
this chapter. Statistical variability due to sampling effects can be 
analyzed using the spectral mixing model and Rayleigh statistics. 

For MEM, measurement noise is modeled as Gaussian noise with a band- 
limited spectrum. In this model, all noise is treated equally so measure- 
ment noise and random processes effects are indistinguishable. Statistical 
variability appears in the estimated autoregressive coefficients and is 
caused by noise and sampling effects. Variability is reduced by taking 
longer sample records. 

Spectrum averaging is frequently used in FFT analysis to reduce the 
effects of statistical variability. In MEM, spectrum averaging and in- 


creased sample length are both helpful. 


7. Fractional period sampling 

All spectral estimators perform poorly if only a fraction of the period 
of a spectral peak is sampled. For DFT (and FFT) analysis, samples less 
than 0.9 Ty are unuSable as most of the amplitude will appear as a zero 
frequency component. For all DFT applications, the data taking method must 
be designed so at least nine-tenths of the period of the lowest frequency 
component is sampled. 

The MEM estimator can give good results when only 60% of a period is 


sampled. This requires severe oversampling and a reasonable low noise 


variance. MEM performance is discussed further in Section G of this 


chapter. 
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8. Spectral strength or magnitude 


Interpretation of the results of a spectral estimate most often in- 
volves the selection of the "most prominent" spectral component. For DFT 
(and FFI) analysis this simply means choosing the highest peak because the 
height is proportional to the amplitude of a sinusoid at that frequency. 

For MEM, the estimate is a power spectral density and the amplitude of 
a periodic component is proportional to the area under the peak. When 
analyzing MEM records, one should avoid the temptation of looking at the 


heights of the components and instead compute the areas for comparison. 


9. Location-of-spectral-peak (LOSP) 


The location or estimated frequency of a spectral peak is another 
important property of a spectral estimator. For FFT analysis, the LOSP is 
affected by the number of samples and the record length and by zero con- 
catenation. If only a fraction of a period of the lowest spectral component 
is sampled, the LOSP for that component will occur at zero frequency. The 
LOSP for MEM analysis is affected by the number of samples and the order of 
the estimate. Typical effects where discussed in Sections G and H of this 
chapter. 

The user mist realize that the LOSP is statistically variable from 
record to record and take this into account when reporting results. If 


many records are available it is possible to get an estimate of the vari- 


ance of the LOSP. 
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10. MEM display resolution 


Since the MEM spectral estimator (Equation 15F.1) is continuous with 
frequency, a digital implementation requires the selection of a display 
resolution, Af. For a very sharp peak it is possible that the output will 
not show the peak because it falls between frequency samples. For this 
reason a practical MEM program must contain a check for missed spectral 
peaks. One way is to numerically integrate S (AE) to see if the total 
power is recovered. Another way is to compute the zero locations for D(Z) 
to see if one lies extremely close to the unit circle. MEM display resolu- 


tion is a practicai problem that must always be accounted for by the user. 
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XXI. APPENDIX I - MATHEMATICAL TECHNIQUES 


A. Integration and Riemann Summation 


A function f(x) is said to be Riemann integrable if in the limit 


b és b-a, bea 
ead Lee: ae 5a ae (AL.1) 


where the interval a-b is divided into n segments with each segment having 
a width of (b-a)/n. If the number of segments, n, is not taken to the 
limit, the Riemann sum will yield an estimate of the definite integral Lope 
The actual value of the estimate depends upon how many seqments are used 
and how the sum is taken. For small n, a good technique of estimation is 
to evaluate the function at the midpoint of the segment for use in the 


summation. The integral estimate using this procedure becomes: 


att ” 
he ee = f(a+ [k-$] b= 8 (A1.2) 


Lop n k= 


Figure Al-1 illustrates this procedure. 

For the purposes of this research work it was discovered that a 
specialized form of the Reimann sum approximation was very well-suited to 
Fourier series analysis. If we assume that f(x) is shifted so that the 
origin is at the midpoint of the independent variable and that there are 
an odd number of segments, the definite integral 


+a 


Te ‘ £(x)dx , (Al.3) 


can be estimated with the following sums 


WN | 
fon = uely fCkbx) ax (A1.4) 
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Figure Al-1. Riemann Sum Approximation for a 
Definite Integral 


t 2a - oon 2a 
aa” GN+1) 2 ** Taneiy (A1.5) 


where: 


k = 0,+1,4+2,...4N. 


(2N+1), the number of segments - assumed odd. 


=] 
W 


Figure Al-2 illustrates this procedure. 

This particular estimator for the definite integral is used many times 
throughout this report. Its usefulness is illustrated in several of the 
derivations involving the estimation of the complex Fourier coefficients 


for digital Fourier transforms. 
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Figure Al-2, Riemann Sum Approximation ysed in 
Fourier Series Analysis 


B. Hilbert Space Vectors 


An infinite-dimensional vector space is called a "Hilbert space" and 
is denoted symbolically by 2 . In ef, the "vectors" are not directed line 
segments, but rather are complex functions of real variables. We define 
a "vector" in * to be a complex fimction, jf, ofa real variable x. In 
other words, a "vector" | is a rule of correspondence which assigns to 
each number x, a pares number ¥(x). The vector space is defined as 
follows: | 

a) Scalars are the set of all complex numbers, c. 


b) The distributive law of addition holds (vector addition): 


U(x) = c, 4, () + Cy ¥, (=) (A1.6) 
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cc) The vector inner product is defined to be 
* 
(454) = i ¥j (&) Vo (x) dx (A1.7) 
all x 


with the following properties: 


(hy st) = Cyst)” (Al.8) 

(24¥4s€p¥y) = €55C¥, 9H) (AL.9) 
% % 

ICH 24) | Ss CH 4) °C» HD (A1.10) 


d) The vector norm is defined as 


G0 =f [ex [Pax (Al.11) 
all x 


and a function }(x) must have a finite norm, (l,}) < 9, ‘to be a 
vector in +. 

e) A set of -vectors fo (x) } form a vector basis set in}. 
The set is said to be complete if any vector in Bl can be con- 
structed from a linear combination of the basis vectors fo (x) te 
The basis set is orthogonal if the inner product (a 5a) is zero 
for all n#m. The basis set is orthonormal if (a »a,) = 6 am 
(the Kronecker delta). 

£) An orthonormal set of eigenvectors fo (x)} are said to form 
an orthonormal basis set in Hw if the set is both orthonormal 
and complete. Any given W-vector, ¥(x), can be written as an 
infinite-dimensional (infinite series) linear combination of the 


orthonormal basis set 
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U(x) = ttn n * = nay oon? yO) (Al.12) 


where the a,'s are the complex expansion coefficients (eigen- 


values of }) defined by the inner product: 


a= (a =f at(xumax (A1.13) 
all x 


The vector norm of (x) can be written in terms of the expansion 


coefficients as: 
2 2 
@wye Bilal (A1.14) 


The theory of Hilbert space vectors was successfully applied to Fourier 
transform theory to give many of the important derivations shown in the 
text. This formulation also can provide useful insight into the problem of 


estimating the complex Fourier coefficients. 
C. Deterministic Functions 


Mathematical functions which are made up of parameters which are com- 
pletely specified for all time and all values of the idiépendent variables 
will be called deterministic. This terminology is used to denote that 
there is no “"randomess” associated with the function. For practical pur- 
poses, a function of time x(t) is deterministic if it can be exactly charac- 
terized by a Fourier series. It is philosophically satisfying to realize 
that an observable function of time, x(t), must be periodic and therefore 
represented by a Fourier series if it is to be deterministic. Time funce 
tions which are not periodic cannot be completely specified because an 


infinite observation time is required. 
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Deterministic time functions are periodic in the time domain and ex- 
hibit only line spectra in the frequency domain. Deterministic functions 
cannot have continuous spectra. A pathological exception to this defini- 
tion exists for single pulses in time. These functions are known for all 
time and have an implied period which is infinite. They also are charac- 


terized by infinite and continuous power spectra. 
D. Noise and Random Processes 


A random process will be defined to be a collection of time functions 
and an associated complete probability description. The entire collection 
of time functions x(t) is called an ensemble of random time functions of 
which the n*® function is called a sample function of the process. In 
general, only one sample function of a random process can ever be observed; 
the other (n-1) sample functions represent all other possible realizations 
which might have occurred but didn't. The following properties of random 
processes are important for our purposes: 


a) Continuous -- Discrete 


In a continuous random process, the random variables x ,(t,) at 
time om can assume any value within a specified range of values. The 
probability density function is continuous and has no Dirac delta 
functions in it. A discrete random process is one in which the random 
variables can assume only certain isolated values. The probability 
density function contains only Dirac delta functions and does not have 


a continuous component. A mixed random process is one in which 


probability density function may have both discrete and continuous 


properties. 
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b) Deterministic -- Nondeterministic 

If future values of x6) cannot be exactly predicted from ob- 
served past values, the process is called nondeterministic. If future 
values can be completely specified, the process is called determinis- 
tic. It is possible for a function to be random over the ensemble but 
not random with respect to time. An example of this type of process 
is an ensemble of sinewaves, x(t) > each with a different amplitude, 
AL? where A, is a random variable. Any sample function of this pro- 


cess would be deterministic with respect to time. 


c) Stationary -- Nonstationary 


If all of the marginal and joint density functions describing a 
random process do eae depend upon the choice of time origin, the 
process is said to be stationary. If these density functions do de- 
pend upon time, the process is called nonstationary. Most real physi- 
cal processes are nonstationary to some degree but quite often they 
can be characterized as stationary for practical considerations. 

A ar is said to be stationary in the wide sense (or covari- 
ance stationary) when the expectation value, E{x(t,) }, is independent 
of the choice of time origin and when E{x(t,)x(tp) } depends only upon 
the time difference, (ty - t,) =. This Puscantecs that the mean value, 
mean-square. value, variance, and correlation coefficient are all inde- 
pendent of the choice of time origin. 

d) Ergodic -- Nonergodic 
A random process is said to be strictly ergodic if the time aver- 


age of one sample function is equal to the ensemble average. It is 
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ergodic with respect to its correlation function when the ensemble 

correlation function is equal to the time autocorrelation function. 
For ergodic processes, mean value and higher order moments can 

be determined by time averages over an infinite interval as well as 


by ensemble averages: 


n a n 1 2/2 n 
E{fx 3 = [ x"p(x)dx = lim a {| xh ceyae (A1.15) 
nt T7020 -T/2 
In this report, most random processes will be continuous, nondetermin- 
istic, stationary, and ergodic. A sample function of the process will 
be called "noise” or a random time function. Usually, only one sample 
function will be available for analysis and its statistical properties 


will be estimates of the "true" values. 


_ 


E. Correlation Functions for a Random Process 


The autocorrelation function Ry(t, sty) of a random process is defined 
as the joint moment of the random variables y(t,) and y(t,): 


+c +o 
E{¥(ty)¥(ty) } = Ri(ty,ty) = i dy, - ¥4VoP(¥, Yq) 4¥, (A1.16) 


The autocovariance, C(t, >t) of the process is defined as: 
E{L¥(t,) - ¥(t,) W¥(e,) - ¥(€2) J} = C L(t), t5) (A1.17) 


For a "wide-sense" stationary process, the ensemble averages defined 
above are independent of the choice of time origin and the autocorrelation 


function becomes 


R,(1) = E{Y(t)¥(t+71)} (A1.18) 
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where T = (t, - t,) is called the "lag time". For stationary processes, 
the time autocorrelation function and time autocovariance function are 


defined as 


1 ate 
Ry(1) = lim tJ yle)y(etndt (A1.19) 
Too -T/2 
and: 
1 ft a _ 
C(t) = lim Ff Cyt) - ye) Lyle tt) - yet 1) Jet (Al.20) 
Toe” -t/2 


For most of the work in this report we will let x(t) represent a ran- 
dom variable with zero mean and then define a zero-mean autocovariance 
function as: 

1 ot/2 
C(t) = lim Ff x(t)x(t+r)de (A1.21) 
T's ~ -~/2 
The autocovariance function is very important in the study of random pro- 


cesses and random time functions because it is used to define the power 


spectrum of the process. 
F. The Power Spectrum of a Random Process 


The power spectrum of a random process is defined as the Fourier 
transform of the autocorrelation function. For a wide-sense stationary 
random process the autocorrelation function is 

1 +7/2 
R(t) = lim =f — x(t)x(t+-)dt (A1.22) 
= ToT syy2 
where x(t) is a sample function of the process. The autocorrelation func~ 


tion and the power spectral density function, S (Cw » (also called the 
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second moment spectral density) are related by the Wiener Khinchine rela- 


tions (Cooper and McGillem, 1971, p. 148): 


= -jwt 

$ (a) = be R (ne Mar (A1.23) 
t= 

R= ., Ss (ude™I aw (A1.24) 


In theory, a knowledge of the autocorrelation function is equivalent 


to a knowledge of the power spectral density function. 
G. Autoregressive-Moving Average Processes 


Stochastic processes which generate a random time series can be suc- 
cessfully modeled as moving average processes, finite autoregressive pro- 
cesses, or autoregressive-moving average processes (Koopmans, 1974, Ch. 7). 
These models can then be used to generate an infinite time series, a dis- 
crete autocovariance function, and a continuous power spectrum. This type 
of analysis is widely used in statistics and is becoming increasingly 
important in engineering for studying the spectra of stochastic processes 
using the maximum entropy method, 


A finite moving average process is modeled by the following series 


mM 
x(t) = Z a, § (t-k) | (A1.25) 
k=-M 


where 2M+1 is the order of the series and the a's are real constants. 
The sequence of uncorrelated random variables, €(t): t=0, +1, ..., models 


a white noise process. The common mean for this sequence is E{E(t) }=0 
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and the variance E{E*(t) }= of. The finite moving average model can be 
interpretated as the weighted sum of 2M+1 random and completely indepen- 
dent impulses, €(t), occuring at past and future times which gives the 
value x(t) at time t. These impulses (called random shocks by Koopmans) 
are most commonly referred to as innovations at time t (Parzen, 1969). 

If sendupsee the requirement of causality and consider models which depend 
only upon present and past innovations, we obtain a one-sided moving 


average: 
2M 


x(t) = wp 8 (t-K) (A1.26) 
The model for a moving average process has wide applicability. In fact, 
every weakly stationary (covariance stationary) process that has : strictly 
continuous spectrum can be approximated arbitrarily close (in a mean- 
square sense) by a finite moving average process. Koopmans (1974, p. 214) 
ahowa that every weakly stationary process with a continuous spectrum has 


an infinite moving average representation. 


The scheme of linear autoregression is a technique that is used to 


approximate the value of a given finite series at a time t by some 
weighted sum of past values of the series. Finite autoregressive proc- 
esses are a class of finite parameter models that may be represented by 


this approach. The finite order autoregression satisfies the equation: 


2M 
seep Pat K) = ECE), (t=0, +1, +2, ...,) (A1.27) 


The boundary conditions for this series are 


bo=l, 5 b.<2 (A1.28) 
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and since the innovations are independent and the x(t)'s and the innova- 
tions are mutually independent, the expectations of the following products 


must be zero: 
E{x(s)&(t)}=0 | for all s<t-1 (A1.29) 
E{§(s) € (t)}=0 for all s#t (A1.30) 


The autocovariance matrix elements for a stationary process are 


N 


defined as: 


E{x(t -p) x (t-k) }=C(p-k) (A1L.31) 


The autocovariance matrix elements can be related to the expansion con- 
stants, Byes and to each other by the Yule-Walker equations. These equa- 
tions are derived by multiplying both sides of Equation Al.27 by x(t ~ p) 


and taking the expectation: 
2M | 2M 
E{x(t-p) = bx(t-k)} = 5 b E{x(t-p)x(t-k)} 
k=0 k _k=0 k 


= E{x(t-p) € (t)}=0 (AL.32) 
p=1,2,..., ™ 


Substituting into (A1.32) with the definition of the autocovariance 
matrix elements gives: 
2M 2M 
> b.C(p -k)=C(p) + & b, C(p -k)=0 (A1.33) 
k=0 k=1 : 
This gives the Yule-Walker equations (bo= 1): 


2M | 
ZS b.C(p-k) =-C(p) (A1.34) 
ket = 
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Equation A1l.32 is similiar to an orthogonality statement about the auto- 
covariance elements and the Yule-Walker equations express a particular 
autocovariance as a weighted sum of the others. 


The variance for the process is defined as the expectation of the 


product of x(t) and §(t): 
2M 2 
Bix(t) §(t)} = Z b.C(k) = (A1.35) 


This equation is used to estimate the variance of the process from the 
autocovariances and the autoregressive coefficients. In maximum entropy 
spectral estimation, this variance is called the error power of the process 
and is minimized by a least-squares criterion. 

An autoregressive series can also be used as a linear predictor. The 
series can be extrapolated one unit ahead in time by using the following 


equation: 


2M 
x(t+1) = €(t+1) - 2 b.x(t+1-k) (A1.36) 
k=] 


The best linear one step predictor (in a least-squares sense) is given by 
(Koopmans, 1974, p. 228): 
2M 


; ; 
x(t)=- 5 bo x(tt+1l-k Al.37 
(ty == Fb, x ) (41.37) 

The one-step prediction error is equal to the innovation variance: 


E{[x(e+1) - x(t) }°} = Efe2(t+D j= 0 (A1.38) 


The autoregressive process can be thought of as high order Markov 


process. That is, the state or value of the series at time t depends only 
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upon the previous states or values. Markov processes are feauceane to the 
study of random variables, probability, and discrete-time random processes 
(Thomas, 1969, p. 114). 

Mixed autoregressive-moving average processes are modeled by a com- 
bination of finite autoregressions and finite moving averages (Koopmans, 
1974, p. 240): 

q rc 
ae d= -p)= = a&(t~k) (Al .39) 
This type of model can be viewed as a linear filter with an output of 
x(t) in response to a white noise input, §(t). The transfer function for 


the filter can be obtained from the z-transform as: 


(A1.40) 


The autoregressive-moving average process can also be modeled by a high- 
order recursive filter. The output of the filter, y(t), is described by 


(Otnes and Enochson, 1972, p. 89): 
(t) 5 (t -p) 7 (t-k) {Al.41) 
y(t) = g x(t-p)+ tek (Ai. 
p=0 P k=1 sd : 


The filter is modeled by the parameters, g, and h,. The complex transfer 


function for this filter is: 


P 
ap ee (A1.42) 
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H. The Power Spectrum of an Autoregressive-Moving Average Process 


By using a filtering model, the power spectrum for a moving average 


process can be estimated by (w= 2nkAt) 
2 
: Ps am -jwk 
S_(a) = oe fe a,e (A1.43) 


. 


where x(t) is modeled as the output of a digital filter with white noise 


input §(t) end a complex transfer function of: 


uM -iwk 
H(u) = 25 ae I (A1.44) 


The spectrum for a finite autoregressive process can be defined from 


the theory of inverting linear filters and z-transforms (Koopmans, 1974, 


p. 217): 
2 2 
i o Pe 
Sw) =f — yak (A1.45) 
2 2 Z 
7 jaca |? 27, LE be JWKFT 
k 


k=0 | 


The spectrum for a autoregressive-moving average process is given by: 


(A1.46) 
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I. Window Functions 


The concept of a window function is very useful in the study of spec- 
tral analysis and sampled data systems. The window fimction serves as a 
mathematical tool to "bridge the gap" between theoretical concepts and 
empirical analysis techniques. Its most important use is to model the 
effect of finite data and finite observation times on estimated autocor- 
relation functions and estimated power spectra. There are three types of 
window functions to consider; the data window, the lag window, and the 
spectral window. 

The data window (also called the observation window) models the effect 
of a finite observation time. A continuous time function, x(t), which may 
hypothetically exist for all time, can be observed for only a finite time, 
Ty This function may be periodic, stochastic, or mixed and its spectral 
bandwidth may be very large but all of its properties must be determined 
from those values in the observation interval, Ty >» which are known. From 
this finite observation, one usually tries to obtain estimates of such 
- characteristic parameters as average value, mean-square value, pes re 
tion function, and power spectral density function. The length of obser- 
vation time (width of the data window) has a profound effect on this 
estimate. 

The inherent process of observation forces all data windows to be 
rectangular functions. The observed data function, denoted by a cireumflex 


accent, is the product of the real function and the data window function: 


x(t) ©h g(tx(t) (A1.47) 
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The estimated properties of x(t) will generally be different than the 
“true” properties of x(t). As a general rule, when the observation time 
ty becomes large, the properties of #(t) become good estimates of the 
properties of x(t) and the effect of a finite observation time becomes 
minimized. This is especially true when the spectral bandwidth of x(t) is 
limited. The effect of finite observations on the amplitude spectral 
function can be seen by taking the Fourier transform of both sides of 
Equation Al.47, The resuiting estimated amplitude spectrum, assuming the 


Fourier transform of x(t) exists, is obtained as: 


1 +o 
X(u) = = f H (we A)x(A) dA (A1.48) 


This convolution of the data window spectral function with the "true" 
amplitude spectrum causes a spectral smoothing effect and a loss in spec- 
tral resolution. Also, the spectral window will introduce spectral mixing. 
A rectangular window causes the most spectral mixing and to minimize this, 
some type of window smoothing may be employed. A data function can be 
tapered at each end to minimize the discontinuity caused by the observa- 
tion. Various types of data windows and an analysis of their performance 
can be found in Otnes and Enochson (1972, p. 281) . 

The lag window is used primarily to model the effect of a finite 
autocovariance function, Ci)» where t is called the lag time or simply 
the lag. In the Blackman-Tukey method of spectral analysis, the lag window 
is also used to reduce the effect of statistical variance by emphasizing 
lag values close to zero lag. The apparent or observed autocovariance 


function, c.() must be finite because the time function, x(t), is observed 
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only for a finite time. An unbiased estimate for the autocovariance 


function has been defined as (Blackman and Tukey, 1958): 


+(T, - |r])/2 
C,(7) = a ff x(t - 3/2)x(t + 1/2)at In| <z, (41.49) 
-(By - [r))/2 


This definition produces an autocovariance function which is even and 
estimated over the lag interval -Ty STS, For statistical reasons, this 
estimate is best for values of lag time near zero. The variance of the 
estimate becomes very large as the lag time approaches +T.. Blackman and 
Tukey suggest using a lag window function to modify the shape of C,(7) so 
that lag times near zero are emphasized. 

The apparent or observed autocovariance function, when multiplied by 
a suitable weighting function (lag window function), is called the modified 
apparent autocovariance fimction and its Fourier transform gives a smoothed 


estimate of the power spectrum of x(t). This modification preduces an 


autocovariance estimator that is defined by the equation 
E(x) = D¢a)C, (1) (A1.50) 


where the lag window function, D(1), may be chosen from a variety of’ 
functions that are commonly used for the purpose of spectral smoothing. 
Lag window finchions have beer very rigorously investigated for their 
properties in an effort to produce a window function which will maximize 
spectral resolution and minimize spectral "leakage". Names for window 


functions commonly used in the literature include; Hanning window, Hamming 
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window, Goodman window, Parzen window, Goodman~Enochson-Otnes window, and 
Bartlett window. Generally speaking, all lag windows have a general shape 
like a cosine bell or a smoothed sinc function. A rectangular window is 
commonly used in theoretical treatments of spectral analysis but usually 
not in practice because it has severe leakage problems. Further discussions 
of window functions can be found in Otnes and Enochson (1972), Koopmans 
(1974), Blackman and Tukey (1958), and Papoulis (1973). 

The spectral window function models the spectral smoothing effect 
produced by a lag window. The spectral window function is defined as the 
Fourier transform of the lag window function. The expected value of a 
collection of estimated spectral density functions is the convolution of 
the spectral window function with the "true" spectral density of x(t). 

The various relationships between the lag and spectral windows and the 


"true" and estimated spectra are summarized below: 


P(f) = F cc t)} (true spectrum) (A1.51) 

B(£) = ‘Filécn } (a single spectral estimate) (A1.52) 

= Fic cnn(n} (A1.53) 

Q(£) = FlD(7)} (spectral window) (AL.54) 
+o 

E{P(£)} = [ Q(E-A)P(ADAA (spectral smoothing) (A1.55) 


J. Caleulus of Variations 


The caleulus of variations problem with constant Lagrangian miltipliers 
and integral constraints is formulated as follows: 


1) Given a definite integral 
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b 
H = [ F(x,p(x))dx (A1.56) 
a 
the problem is to maximize H with respect to the function p(x) where 


p(x) has N constraints as given by: 
b x 
J (spa) dx=G, k= 1, 2,3, 22 N. (A1.57) 
a 


The integrals mst all be finite and the limits of integration are 
over all defined values of the independent variable, a F(x,p(x)) and 
Q, Cx,P(x)) are explicit functions of x and p(x). p(x) is a function 
of x alone. 

2) Substitute for the function to be maximized (in this case p(x)) 


the following modified function which has N arbitrary "perturbation" 


functions: 
B(x) = P(x) + €,7, (2) + eT (%) + oo. QTC) (AL.58) 
N 
B(x) = P(x) +E 6,n,@) (A1.59) 


This produces a function with p(x) and N additional degrees of freedom. 


3) Next, form the following auxiliary function 
N b 


where %i=% Pe), €17 > €5%|> eco Thy) + 
4) wis minimized with respect to eack ¢. by evaluating the following 


derivatives at all €, = 0: 


be, 


3 | QW 20 
ll ¢ =0 all ¢=0 
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mere SB }.0 (Al. 61) 


all ¢.= 0 


This procedure will yield N equations that can be used to solve for 


N constraint constants. The partial derivatives can be expanded by 


noting that: 
A A 
SD QP 
ep a, ap OF 
af 
3, - n(x) (Al. 62) 
All the partial derivatives become: 
ae r col 
= dx 
7 =| 
all e,=0 all = 0 
N b Qa wo 
+ DTA “|e -— (A1.63) 
mi ™ ! oe, =a ‘] 


all = 0 all ¢.= 0 
The partial derivatives for F, Q and o, are written as follows: 


= 3, F(x,p(x) + e+ oT, + ee | 


cose 
2, 
se a all ¢,=0 
FD oF | 
2 * 2 
all ¢ =0 all ¢,=0 
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Sn = 7,(x) i (A1.65) 


ep 
all e.=0 all e.=0 


—=0 (Al. 66) 


b N Qa 
aw oF 
co ae = (x) [= +2 AL. —] ]dx=0 (Al. 67) 
= J The A mt m ap | | 
all 6 = all €.= all ¢. = 0 


5) Since the integral of Equation is zero for any arbitrary "pertur- 
bation" function ny, ()s only two possibilities exist. Either 7, (2) =0 


which is the trivial solution or else: 


N 

S—) + BA, ia | = 0 (A1.68) 
op (x) ep (x) 

all ¢,=0 . all ¢.=0 

Equation Al.68 is called the Euler-Lagrange equation and may be 


written in an alternate way as: 


N : 
—2—fr(x,p(x)) + E Aga) I =0 (A1. 69) 
op(x) = all ¢ =0 


The Euler-Lagrange equation will generally contain N Lagrange mlti- 
pliers, the independent variable x, and the dependent variable. p(x). 
6) The Euler-Lagrange equation is solved for p(x) in terms of the 
4/8 and x. This solution will generally be very complicated and 


quite often transcendental. The explicit solution for p(x) will he 
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represented by V(x, Ay» Ao» wes 4D - This solution is then substituted 


into the N constraint equations 
b 
J %C%,VGRAy Ay, oo Ay) d= aL (A1.70) 
a 


and the N equations are solved for the N values of An? the Lagrangian 


multipliers. 


7) ‘The solution is completed when these a's are used in the explicit 


equation for p(x) to give: 
P(X) = VORAq sda, oo AY) (Al.71) 


The calculus of variations problem with a continuous Lagrangian milti- 
plier function is formulated as follows: 


1) Given the definite integral 


b 
H = f F(x,p(x))dx | (A1.72) 
a 


the constraints are now assumed to form a continuous function and the 


constraining equation becomes 
: | 
J 0C1,x, p(x) dx = (9) (A1.73) 
a 


where the continuous parameter tT replaces the index k. 
2) Substitute for the function to be maximized, the following modified 
function which contains an arbitrary "perturbation" function n(7T,x) and 


an ¢(1) continuous multiplier that is independent of x. The modified 


function becomes: 


p(x) = p(z) + [ea n(t,x)dt (A1.74) 
all + 
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Note that the interval of +t for $(1) must be the same as that for 


e(t) and 7(1,x) or there will not be enough “degrees of freedom" in 


the product e(1)n(1,x). 


3) Next, form the following auxiliary function using p(x): 
b A 
w=H(e,7)+f A(t) Cf Q(t,x,p)dx - 6(1) Jat (A1.75) 
all + a 


4) Calculate the partial derivatives of w with respect to e(1T) and 

evaluate this partial derivative at e(T)=0 to minimize the function 
A 

w at the point p(x) = p(x). 


The partial derivatives are expanded as before: 


ow - Dp 
3e(7) ~ ah BeC7) ist ®) 
e(r) =0 e(r) =0 
A 
E - Hp 
ae(T) e ap ae(T) ABE d) 
e(T) =0 e(T) =0 
A 
-RD_ 
ae(7) A 2e(T) | 2 At278) 


A 
It is important to note that the partial derivative of p(x) with 


respect to €(T) can be reduced to a function of x alone as follows: 


A 
a = mee [p(x) + f e(n)yn(r.x)4r]| 
e(7) =0 all T e(7) =0 
% an 3e(1). 
eo ae(t) * 7 Becry 87 
e(T) = 0 
= [ n(t.x)dr (A1.79) 


all t 
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The integral of 7(1T,x) over Tt leaves a function of x alone, therefore: 


A 
2ereh = f n(1,x)dt = K(x) (A1.80) 
8 e(=o 2th t 


With these various partial derivative expansions, the partial deriva- 


tive of w becomes: 


: —— 
ae(T) 


b + 
+f fac en dxdt (Al.81) - 
e(t)=0 ? e(7)=0 v= 


all 


Rois 
ze(T) 
Combining the integrals and reducing gives: 


b 
=f x~ & 
e(7)=0 ? Pe(1)=0 


+f xan 8 
bate =p 


dtldx=0 (A1.82) 
e(71)=0 


cn 
3e(T) 


5) Since the integral is zero for every arbitrary "perturbation" 


function K (x), the only nontrivial solution is: 


A A 
HE =.P@)) +f ay BC ze GD) ar=0 (A1.83) 
ap(x) all t op (x) 
e(7)=0 - e(1)= 


Equation Al.83 is the Euler-Lagrange equation for a continuous con- 
straint function defined by an integral constraint. This equation 
will contain a continuous Lagrangian mltiplier, 4(1t), the independent 
variable x, and the dependent variable, p(x). 

6) The Euler-Lagrange equation must be solved for p(x) in terms of 
the mltiplier function A(T) and x. In general the solution will in- 


volve a transcendental equation in p(x) . An explicit solution would be: 


p(x) = V, @sA(1)) (AL. 84) 
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7) This value of p(x) is then substituted into the constraint 


equation to give: 
b 
J t,x, 0, ACT) dx = (7) (A1.85) 
a 


This integral equation mist be solved to yield A(t) as some function 
of ¢(1) represented by: 

ACT) = Ry(OC7)) (A1.86) 
Again this solution may be transcendental. 
8) The solution for p(x) to maximize H is completed when the solution 


for ACT) is used in Equation A1.84 to give: 


p(x) = V, (xR,) (1.87) 


418 


XXII. APPENDIX II - FOURIER ANALYSIS 
A. Fourier Series - Equivalent Mathematical Forms 


A continuous function f(x) is periodic in x if it is defined for all 
real x and if 
£(x+nT) = f(x) (A2.1) 
. 
where T is called the period of f(x) and n is any positive or negative 


integer including zero. 


The familiar Fourier series representation of f(x) is given as: 
- = 2™m -. 2m 
£(x) = at 2 (4,008 — x + b sin a x) (A2.2) 


The Fourier coefficients for (A2.2) are computed by the following integrals: 


1 +T/2 
a,"F ta ee (A2.3) 
+T/2 
2 2rm 
a= £(x) cos(=— x)dx (A2.4) 
n T Jp T 
+T/2 
b, * 2 s fo sin( x) dx (A2.5) 


The complex representation of a Fourier series is obtained by using 


Euler's equation, eJ? = cosg + jsinag, to replace the sin and cos terms of 


(A2.2). When this is done, the series can be written as: 


: ao +)" 
£(x) . k, + + [ke +ke 7 (A2.6) 


419 


The complex coefficients are related to the a's and b's as follows: 


7 ee ae ae 
k= 8, x, | = 5 (ajtb)) 
(A2.7) 
1 aD 
ko aS (a= j bD o, = Arctan Ca 


The complex series representation is most often seen in the following 


compact form 


+0 +j Bx 
£(x) yee ce (A2.8) 
c_=z J. f@e dx (A2.9) 
-T/2 
withe =c. 


Another common representation of the Fourier series is the phase angle 
representation. This representation is obtained from Equation A2.2 by 
using the substitutions a_=dcos¢@ and b_ =-d sing. After several 

n n n n n n 


vignometric manipulations we obtain: 
£(x) = a, +4 d cos a x+@) (A2.10) 


The values for d_> and $, are computed from the a's and b's as follows: 


= (a2 4b°y% 
dq. = (a, +b.) 
(A2.11) 
-b 
¢, = Arete) 


The relationships between the complex representations and the phase 


angle representations are summarized below: 
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= * a oe 
a) a =k +k, k= la, 7 db) 
A2.12) 
ot = ee . 
b= idk,-k) ha penta 
i ass c= $a,- ib.) 
ay i; oa -n n - 2° >n J na 
1 (A2.13) © 
b= i(e,-¢_)) c= fat ib) 
lc | = 4g? 4.425% $ = ketene =a (A2.14) 
n 2>n on n = : 
5 ee 
c) a= d_cos bo qs (a +b.) 
->, (A2.15) 
db = -d sing o = Arctan(——— >) 
n 
c = d etd On d= 2\c 
n 2n n a 
Lg cits rale,) oak 
c,=5 de 6,7 Arctanlzete_}! 


Each representation of the Fourier series has its useful applications. 
Quite often the phase angle representation is used in discussions of net- 
work theory and wave propagation. For data analysis, the compact complex 
representation is used because it is identical to the digital Fourier trans- 


form and also because it is notationally convenient. 
B. Fourier Series - Discrete Independent Variable 


An infinite periodic time series can be represented by a digital 
Fourier series. Using the compact complex representation and assuming 


equally spaced x-vaiues we get: 
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+M +j Prax 


£(kAx) == ce / (A2.17) 
There are only (2M+1) values of Co because for a finite number of values 
of £(x) in the Fourier period, T, only a finite number of complex ampli- 
tudes can be uniquely specified. This result points out the important fact 
that a digital Fourier series must, of necessity, be spectrally limited. 
_ The complex amplitudes can be exactly specified by the following digital 
routine which is obtained by using a Riemann sum approximation for the 


integral of (A2.9): 


.2 
Z MM -j <P akax 
car keaiyt (KAx) € (A2.18) 


Equations A2.17 and A2.18 form a digital Fourier transform pair if the in- 
crement Ax is such that Ax = T/(2M+1). The c's calculated from (A2.18) 


can then be used in (A2.17) to reproduce the exact f£(kAx)-values. 
C. Fourier Transform Pair 


The Fourier series can be used to derive the Fourier integral and 
Fourier transform. If a Fourier integral is to exist for an arbitrary 
function f(x) (not necessarily periodic) the function must be piecewise 
continuous on every finite interval (Riemann integrability), it must have 
a finite average value, and it must be absolutely integrable (sufficient 
but not necessary). 


The Fourier integral may be written as 


£(x) = 4 jam cos wx + B(w) sin wx dw (A2.19) 
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where: 


to 

A(w) = [ £(u) cos wu du . (A2.20) 
+2 

B(w) =[ £(u) sinwu du (A2.21) 


“co 
The Fourier transform is the complex representation of the Fourier integral. 


Using Euler's equation, the Fourier integral becomes 


+00 . +o 2 
f equye™ au te 0 equye I aulaw (42.22) 


to 
£(x) = plea 


which can be reduced to: 
Je <® ae wx, sae -jwu 
£(x) ae e Cf f(u)e J du |dw (A2.23) 
. -—2 


“Oo 


Equation A2.23 is used to define the Fourier transform pair: 


+>. | : 

£(x) =5— fe *Ecuyaw (A2.24) 
Fo apse . 

F(w) =[ £(x)e ~" dx (A2.25) 


Many functions that appear in physics and mathematics are form lated 
in terms of the Fourier transform and thus are associated with Fourier | 
transform pairs. Two common examples are: 1) position and momentum in 
quantum mechanics and 2) autocorrelation and power spectral density in 
Signal processing. The autocorrelation function, R(T) of a random process 
and the power spectral density function, S(w), of that process form a 
Fourier transform pair: 


eee 
s(w) ={ R(r)e Far (A2.26) 


=o 
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T= 5 J S(ue Taw (A2.27) 


An alternate mathematical form that is often seen is obtained by substi- 


tuting 2nf =w to give: 


+o . 

P(f) =f R(r)e I" Tay (42.28) 
+o - 

R(t) =f P(e)e IF Tag (A2.29) 


D. Digital Fourier Transform 


The digital (discrete) Fourier transform is obtained by using a 


Riemann sum approximation for the integral of Equation (A2.24): 
pee a +jnawkax 
£(kAx) = 5- Aw & F(nAw)e (A2.30) 
2m nzeN 


The spectral amplitudes are obtained from the sample values of f£(kAx) by 


the Riemann sum: 
to = 
F(naw) = Ax £(kax)e jnawkax (A2.31) 


Other mathematical forms of these equations are often used to develop the 


theory and practical application of digital Fourier transforms. 
E. Special Functions 


There are several special functions which occur repeatedly in the 
mathematics associated with Fourier analysis. Three of the most important 


are the Kronecker delta, the Dirac delta and the siuc function. The fol- 
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loving tabulations of the various forms of these three functions are in- 
cluded because they have proved to be valuable for use in a variety of 
derivations. 

The Kronecker delta plays an important part in Einstein summation 
notation and in the formlation of digital Fourier analysis. The Kronecker 
delta is defined as follows: 


1 k=n k,n are integers 
6, = (A2.32) 
a | a 
Some of the various mathematical forms for the Kronecker delta function 


are: 


— Sinn(k-n) 
Sin ne =n) (A2.33) 


40/2 44 Ai (ken)x 


; ; 
6 iced e dx (A2.34) 
kn T 80/2 
ae 
1 +40(k- 
cam a J e jek 2) 39 (T = 27) (A2.35) 
g*/2 a Be 
a.” ay cos (> kt) cos e nt)dt (A2.36) 
-T/2 
27 
aye mP 
Sin” yar MFI S (42.37) 


This last form was discovered in the derivation of the spectral mixing for- 


mula and can also be written alternately as: 


wae ae ae 
84m CES aie mei n)p] (42.38) 
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The validity of this equation was shown using a double precision computer 
program, The magnitude of the indexes was shown to be restricted to the 
ranges; 0SksM, OsnsM. 

The Dirac delta function (or impulse function) plays the same type of 
mathematical role in continuous functions as the Kronecker delta does in 
discrete functions. The Dirae delta is defined as follows: 


C) xy 


8(x-y)= (A2.39) 
0 otherwise 
‘i §(x-y)dx=1 (A2..40) 
allx 
J £00)8 @ - y)dx = £(y) (A2.41) 
allx 


Equation A2.41 is the sifting (or sampling) property of the Dirac delta. 


The Dirac delta function also has the following integral definition 


(Cauchy principle value): 


+c 
b(x-y)= f etm wha (42.42) 


oO 


The Dirac delte can also be represented in the following mathematical forms: 


“ = sin nT(x -y) 
&(x-y) La aT (x-y) (A2.43) 
2 2 
8(x-y) = Lin te (x-¥) (A2.44) 
S(xty)= Lim Sea (2.45) 
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For conceptual purposes, the Dirac delta is often represented by a rectan- 


gie function of height T, width 1/1, and unity area: 
8(x-y) = Lim T rect {T(x - y)} (A246) 


The sinc function appears in most discussions of Fourier analysis and 
sampling theory. It is the Fourier transform of a rectangle function and 
can be used to approximate a Dirac delta function. The normalized sinc 


function is defined as 


sine) = Sine (A2.47) 


and has the following integral definitions: 


7 54 
f SiR TX gy =] 


1X 
+t | w/2 3 m>0 
i Sane dx = (¢] n=0 (A2.48) 
0 _ Low/2 n<Q 


A more generalized form of the sinc function can be written as: 
= Cees 
AT sinc(r) = AT. - (A2.49) 


Figure A2-1 show a plot of the generalized sinc function. The peak ampli- 
tude of AT. occurs at wW* W The zeros of the sinc function occur at 
w= wy, tn — n=1,2,3... and the bandwidth between first oulls is 
m 
BWFN = 2n/T,. - 
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o 

eo o 4, 


Figure A2-1. A Generalized Sinc Function 


The sinc function has the following useful mathematical properties: 


+> sin(we w)T.. 
1 ar 8 -4 A2 
an J Ate gaye 2 | pe 


sin(w- wT 
ao Atl arar = TA 6(w- w,) (A2.51) 
me om 
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PROGRAM 03 


PROGRAM 04 
PROGRAM 05 
PROGRAM 06 


PROGRAM 07 
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COMPUTER ALGORITHMS 
- Discrete Fourier Transform Testing 
- Estimated Kronecker Delta Function 


- Estimated Spectral Amplitudes using 
Kronecker Delta 


- Discrete Fourier Transform Analysis 
- Fast Fourier Transform Spectral Density 
- Estimation of Single-Sinewave Parameters 


- Maximum Entropy Spectral Analysis 
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tjCe "RUSSLUL %s TIMCHSOSFACELS=7E 
¢ 
c FFCGFAMY O1 
c 
Cc OISCRETE FOURIER TRANSFUCEREM TESTING 
Cc 
Cc TRIS PROGRAM IS UStl TC ANALYZE THE EFFECT CF 
CG VARICUS SAMFLING CONCITICNS CN THE CISCKETE FUUFIER 
Cc TPANSF ORM ESTIMATE CF TRE SFECTFUM CF A TIME SERIES. 
Cc THE INFEUT TIME SERIFS IS GENERATED IN THO PRCGFAM. 
Cc 
OLMENSICN FVALLUF (2C1) »FX( 804) 2464090) 03 (600) 
Cc S& i=1,.400 
ACT) =CeS 
5 fiC€1=00e0 
Cc 
Cc INPLT DATA VALUES 
¢ 
4(1)=4.0 
6(3)=1¢40 
Cc 
TF=1.0 
TIN=301 
N3S=e2l 
FNS=NS 
TISLZETNSFNE 
SCAL Z=200 
¢ 
NLS=NS 
NMAX= (NS—1)72 
DOC V=00e0 : 
C 
¢ GENERATE CATA VALUFSSs FQLATION( 12021) 
CAlLt FSRSCFXeACE eCCVeKRSEeTOEL TF) 
FRINT G 
Sg FERMATCEL® ohlXs® Ko oF Xe*TIME® -13Xe*F( TIME) *s/) 
CO 10 I=1eNS 
FVALYE CIJ=FXCI) 
IT=I—-NMAX—1 
XQEL TA=IT IS TCEL 
10 PRINTe TI esXCELTASFX(I) 
¢ 
c CGMOUT=S COMPLEX FOURIER AMPLITUDES. EGUATION(120e>) 
~ 
CALL LNSFTWMCF VALLE sh et eALS eoCCV) 
PRINT 34 , 
34 FGRMATCELS aX 5 ON 8, Xs OCHVALUE 8 9 4X0 9 CMAG Sg BX 


Bo SCPYHASE 80 EXe *FRECHHZ* 6 7Xe* CMAGC-CB'S ) 


27° 


2a 
es 
30 
31 
22 
33 
34 
35 
36 
37 
3é 
39 
40 


41 


42 
43 
44 
45 
4E 
a7 


4A 
4S 
50 
S1 
$2 
53 
54 


5S 
S6 
57 
38 
es 
é0 


61 
62 
é2 
64 
65 


AwnNnaAaaN 
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AL MAX=(NLS-1)/2 

CG 35 T=lsehLMAX 

Kazi 

CMAG=C CACTI) *ACT tC i dee 1) 24 eS Ze 
C¥=CMAG/SSCALE 
CMSGCS=ZO0e*AL OCI CCCM) 
BTCSTHECTISACT) 
ATESTH=ATEST#ATEST 
ATESTH4TEST##*0O0S 
AT=1U000e-ATEST 

TFCAT) 202620321 


20 CFHASE =- 2899989E 
Gb TO ze 
2l CONTINUE 


BA=-BCTIISACI) 
CFPrASE=ATANC BA) 
CPHASE =CFHASE #57 e 25577 


FNLS=KLS 
22 FREOQ=XI/S(TCELFFNLS ) 
35 OSINT 36e0eI sDCVeC MAG sCFRASE oFFE Ce CMAGER 
26 FURMATCI6s 4E 1243 6F LOeZ) 
c 
Cc COMPUTE Tre ESTIVATEC TIME SERIES», FCUATICN(1L2Cef&) 
Cc 


NS=4 «NS—-3 
TDOCL=TOEL/S4 
NMAX=(NS—1)72 
TF=TN 
CALL FSRSCEX sALE OC VERS eTCEL eo TF) 
PAINT SO 
So FORMAT C® Le eLixs* Ko 7Xe *TIME* el lx 
Te fSSTIMATEC FC TINE) 7) 
oC 160 T=1eNS 
Tl=I-\M¥ex-1 
XCEL TA=1i4 TIOEL 
100 PEINTs TIexXCELTA,FX( I) 
Sol £T7P 
END 
LINE SPECTFUM SUBROUTINE 


THIS SUBROLTINE GENES ETES THE FCUFIEF CCEFFICIZENTS 
A ANC @ FCR NUS SAMPLES OF THE FUNCTICNe FVALUre 


SULBRILTINE LNSFTMCOCFVELLUE eAebd eALS el CV) 
CIMENSICN FVA4LUVUE (201 )24(400)-8(400) 
NLMWAX=(ALS—1) 72 

FI=3-1415S cE 

XKNSNLS 


82 


as 


2s 
38g 
20] 
$1 
S2 
$3 
G4 
es 
S6 
c7 
98 
29 
100 
101 
102 
103 
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DC V=O65 
CO 15 J=1eNLS 


1S CCVECCV4eFVALUE (J) | 


DC V=DCV/XNA 
SC 15 J=1leNt5 
16 F VALJE (J) SFVALUE (J )-CCV 
O03 10 K=1leNLWAX 
ASIIM=H0 6 
8SUM=06 
XN=N 
cc 5S J=1leNLS 
JJZ=J—-NLMAX—1 
XJI=IJ 
A&G=2 e OFFI FXNFXIISXNN 
ASUM=CCS(ARGIBFVALLE (J) 4aSU¥ 


5 BSUM=SINCARG) €F VALLE ( J) #93SUM 
ACN) =2 eO#ASUM/XNN 
10 3ON) 220e0*2 SUMSXNA 
99 17 J=1-eNLS 
17 F VAL UE (CJ) =FVALUE (J) 4C0CV 
R& TURN 
END 
Cc FCUPIES SERIES SUEFCUTINE 
c 
c THIS SUBROUTINE GENERATES SAMPLES OF THE CATA 
c FUNCTICNes FXCI)»s LSIAG THE FOURIER CCEFFICIENTSe 
c ACI}»e BC1)~e AND OC-VALUE*s CCVe AND THE NUMBER OF 
c DESIRED SAMFLESs ASe TOEL IS TRE SAMPLING PERICO 
c AND TF IS THE FCUFIER FERICC. 
c 


SLSRTUTINE FSRSCFX pAcE sl CV eNS STOEL o TF) 

DIMENSION FX(@04).AC 400) .B( 400) 

NMAX={NS-1 972 

PI =321415926 

XNS=NS 

cc 19 T=1.AS 

K=]—-NMAX—I1 

XK =K 

EXK=%e 

00 S N=1+NMAX 

XN=N 

AnRG=(2eOAFIKEXACXKETCEL D/STF 
5 FXK=F XK tAON) PLOUSCARG) BON) SSIA CAG) 
10 FX(I)2= FxXK + CCV 

RE TURN 

END 


SENTRY 


w~ 


Z 


ON ONO DOW VD 


K TINE 
-10 ~Oel47ELSOS O2 
-g —001328579%EFE O1 
-@&8 —-O-el1B0SS25 OL 
-7 ~JelLCIZZIZE Ol 
—¢é ~O0e8ES5T71I41E 09 
-—- —Oe73E0GSOE CO 
—4 —“O0«eS3904760E CG 
-3 —“0e442ES70E 00 
-2 ~O04.2S5S238CE CO 
<1 —Oe«1476190E O90 
0 CeNDGCCVOVQOE O00 
1 Ool4?76LSOF O00 
2 06«23952380F O00 
3 Oe442ES7CF CO 
4 OeS7O47U0E CO 
S Oer 3809G5S0FE O00 
G Oe5FS7141z O00 
7 GelLOS33ZB3Z3E Ol 
@ OollLEOSS2E O11 
S Jel 32S5S7OE QI 
10 OelS7VELGOE Ol 

DC~VALUE CwaG 

~O0-e132E 00 Oelsocr vd 
~O50132E 00 O«222E O00 
—Qel3ee 00 Oe200c O17 
—Oel3SeE vo O-1a6F CC 
—00e132E 09 Oe 741E-91 
~OelZ2e 00 Oe427E~-ul 
—0e132E 00 0 -363E-vV1 
—Oel32e— co Oe647E-01 
—Oel32E O00 Oes80E OC 
—Oel32e 00 Oe261E 00 
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FqTIime) 


~CeAMJESES2CE 
—-Oel&O0S7ESE 
00 1947460E 
CoSle4E2 iF. 
Oe 28&46E33E 
~Cel273E04E 
—CeZS7SSSVE 
—0 046255336 
—Ce4E27€02t 
0 eZ047SG1L1E 
De4000000E 
Cec t SQOEESF 
—-Oel77SE44E 
~—Ce cd EE4336E 
—CeS ZELEISE 
Net 7S9SE77E 
CeZ177713€ 
0 e4S003E6EE 
O00 141S427F 
~CelSESLISE 
~00e3521418EF 


CFEASE 


cc 
00 
co 
Ol 
OL 
02 
G2 
02 
Ce 
02 


—Oe424EF 
-0eSS3E 
-~O-1CSE 
OslE4E 
Oe7lEE 
Oe2C7E 
0 e4EAE 
Oo?77LE 
0 «8&ee 
~0e«@95E 


cl 
C2 
Oo} 
cl 
Ol 
cl 
cl 
Ol 
oc 
O1 
01 
cl 
om § 
Cl 
C1 
co 
O1 
01 
Oi 
Cl 
01 


FREQ-—FZ 


Ce 323E 
0e64S5E 
CeS68E 
Oel ZSE 
OelSl1E 
0e1S4F 
Oe2et=e 
Oe258E 
Ce2sce 


Oe323€ 


joke) 
co 
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Cl 
o1 
o1 
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O1 
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CMA G—D6B 


~22e71 
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TIME 


—90«1476190t v1 
—~Cel43S285E Cl 
—Vel402380F O1 
—Cel 36S476E 01 
~Oel32ce7oe Ci 
~Oel2GisSé66EF Cl 
—Oel2S4761E Ol 
~OelZl7PSEE Ci 
-OellSO9S2E V1 


/—~Oell44047E 01 


=OCellO71I42E O1 
-FelIVOZ37F Vl 
~O4e1033333E O1 
—0 e9964283E 00 
—00eS59S236E 10 
—~O04e9226128E 06 
—-O eEBS7I4GIE OO 
—O0e848R80S3E 00 
“Oce@1ISU45E 00 
~-O«77499SBE 90 
~O4e7380SS5SOE OC 
-Oe7O01LGO3ZE 00 
—-0e6642855F 90 
~Oe6273EC8E a0 
~OeS304760E v0 
—-0e5535713E 00 
-OeS1EGEESE CC 
—-9 e4”"GS7TEIGE OC 
—~OeS42ES70F 00 
—-0e4059S23E€ 30 
0 e369047S5E 00 
~00e23214Z8E 0 
—-0 e2952380E 00 
“0 e2583333E OC 
—~9eZ2142ESE CO 
~Oel84S238F 00 


“=0014761S0E 00 


~OellO7L43E OC 
~0«73809S0E-0i1 
—O5«3ES0475E-C1 
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0e369047S5SE-01 
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ESTIMATEC F(TIME) 
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CeotEGE7TSEE Oi 
Qe27S8279E O11 
Oo lSVEP7SIE Cl 
~O«e«S1O77G623E-O01 
~ONel1l2TZIE&4E Cl 
~QelS?7SSESE C1 
-Oe223E793E 01 
~Oe227US0SE Ol 
~Oe237SSICE 01 
~Oe27749ESE O01 
—~Ce3assod2se 01 
~VeGISOBESE Cl 
—0«e4625520E 01 
~Ceo444EECSE Ci 
—9eI354721Et Ol 
~Ce2CS3671F& Oi 
~CeMES734SE OC 
OeG180411E 00 
Oel?ES8@14E O1 
Oe20E7ESSE Ol 
Oe20475SS5SE 01 
CecCES4202F Cl 
Oec400G2SE O02 
Oe SLL61OSE 61 
De2SSSS74E C1 
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$STOP 
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‘JOs "RUSSELL S'eTIME=19.PAGES=50 
Cc 
Cc PFOGrAM 32 
c 
ESTIMATED KFONCCKER Lee LTA FUNCTIUN 
= 
re THIS eCUTINE CVALUATZS Tei APFRIXIMATIGN TO THe 
S KPSONECKER Och TA FUNCT Tote CGUSBLE PFLCISIOGON IS USE 
ae TC KESCLVE THE EXACT VALUCS OF The GFF-DIAGSNAL 
c SLEMENTS © , 
Cc 
REAL ®E THITA AMAT sCALTCR ol NN OGet KoTF oT NeOKeF I eCCUS 
INTEGEP Ost 
C INPUT CATA 
N=5 
=Ss 
TN=le 
TF=le 
NN=2Z aN] 
ONN=NN 
MM=2aM4] cf 
PI =3e14159265358S97 
c 
FRINT S5 
3s FURMATC8El® SX ot Qt oTX etn hs TXe Otc TOR" s16Xs *DELTOF #7) 
06 8° T=1+NN 
aVaT—eNnw-l 
be=Q 
CG 8S L=15MM 
R=L-M-1 
DR=RK 
AMAT =900 
co SO K=1eN 
CK=K 
THE TA=Z2 9 PPI ROK*T NG COGS TNOEDEFSTF IZ ONN 
boy 8] AMAT=DCOSC THE TA) AMAT 
CELTOR=Clet2e AMAT I/SONN 
EQ PRINT E2sQo0k cELTORn sl L Tur 
oi FORMAT (Id el60F10 eSei 25016) 
STOF 
END 
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-—02e00000 
0 200000 
-~0200000 


(e] ARnANR 
evvuveu 


1200000 
Qe 00000 
0 200000 
0200000 


—-0200000' 


0200000 
—-0200000 
0200000 
-0 200000 
0200000 
~0 200000 
0200000 
1.00000 
0200000 
-0200000 
0 200000 
—0200000 
0200000 
-0 «00006 
0200000 
—0 200000 
0e00000 
—0e00000 
0200000 
1200000 
0 «00000 
—0 000000 
0e 00000 
-0200000 
02 00000 
—02e 00000 
0200000 
—-0e 00000 
0200000 
—0200000 
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DELTGR 


Oe 10000000000000000 021 
0 e 3024600771859489D-13 
—O«0 31389032787129430-13 
0 33723024372989130~13 
0 3740442299328028D-13 
Oc $321543123353422D-13 
—0 65189787993293232D-13 
0e 66156676096924S6D0-13 
— O00 $027 122483861272D0-13 
0+ 1430 421437863639D-12 
00 3630101422299677D-12 
00 30246€0077185S4890~-13 
001000000000000000D 01 
Oe 3024 €00771859489D-13 
—0 0313890327871 29430-13 
Oe 3372 3024372985130-13 
—06«6 3740442269328028eD~-13 
0 0 4321543123353422D-13 
—00 5189 7879932932320-13 
0 ©6615667609692496D—13 
~0 09027 1224838612720-13 
Oe 143042143786363590-12 
—0 e 313890 3278712943D-13 
Oe 302460 0771 8554 890-13 
Oe 10000000000000000 01 
003024600771 859489D- 13 
~ 00 31366032787129430-13 
0 6 3372 2024372589130-13 
0 0 3740442295328028C-13 
Oo $32 1543123353422D-13 
—0 0518978799 3293232D- 13 
02 661566760965 24960-13 
— 00 9027 1224838612720-13 
0 0 33723024372989130-13 
—0c« 313890327871 29430-13 
06 302460077 185$489D- 13 
0e10000000000000000 01 
Oo 302460077 1 &5$48S0-13 
—0 e 313890 3278712943D- 13 
Oc 33723024372989130~13 
—0 0 37404422963280280- 13 
00432154 3123353422D~13 
— Oe 51 8S787993293232D-13 
0 e66156676096924960- 13 
O00 3740442299328028D0-13 
Oc 33723024372989130-13 
—0 6 313890 327871 29430-13 


WUUWWWHUUNNNNNNNNNNNe eee eH OOOOOCOOD000N0 


-2 
-1 


| 
Hm NW 


WUD Ee WNHmo 


0e00000 
1.200000 
02-9000900 
~Oe«e 00000 
0.00000 
~02e090000 
0200000 
-—0200000 
0200000 
—-0200000 
0200000 
-0200000 
02 00000 
1200000 
0200000 
- 0200000 
0200000 
—O2 00000 
0290000 
-0 200000 
0-900000 
—-0 e00000 
Ge90000 
-0200000 
0200000 
1200000 
0200000 
—-0 200000 
0200000 
-0 200000 
0200000 
—-0.600000 
0200000 
-0200000 
0200000 
—0 «00000 
0e900000 
21200000 
Oe 00000 
~0200000 
0 «00000 
-—0e00000 
0 «00000 
~0 200000 
0e 00000 
~0 «00000 
0200000 
-—02e00000 
0200000 
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Oe 3024600771 8594890-13 
O0«19000000000000000D O1 
0 e 3024600771859489 D- 13 
—O« 31389032 787129430-13 
0 e 337230 24372989130-13 
—O0e 3740442299328028D0-13 
0 04321543123353422D- 13 
—005189767993293232D-13 
Oe 43215431233534220-13 
~0 ¢ 3740 4422993280280-13 
Oe 3372 302437298913D-13 
0 0313890 32787129430-13 
Oe 302460 0771 8594890-13 
Oe 10000000000000COLD Ol 
0 e3024600771 8594890-13 
— 00 31389032787129430-13 
0 e 3372302437298S 130-13 
—0 e 3740442299328028D-13 
Oe 432 15431233534225-13 
—9 05189787993293232D— 13 
0043215431 233534220-13 
—00 3740442299328028D-13 
0 e 33723024372989 130-13 
— Oe 31389032 78712943D-13 
0 «30 2460077 185S48SD- 13 
02 1000090000000000D 01 
Oo 3024€00771 8594 8SD-13 
0 ¢ 31.3890 32787129430-13 
Oe 3372 3024372989130-13 
— 00 37404422993280280—-13 
0 e66156676096924 960-13 
~%a $1 897879932932 320-13 
0 e4321543123353422D- 13 
~—00 374044 22993280 280-13 
Oc 3372302437298S5 13D—13 


.70e 313890 32787129430-13 


Oe 3024600771 8594 890-13 
OelO000000000000COCE O12 
00 3024600771 859489 0-13 
— Oe 3138903278712943C-13 
0 e 337230 24372989130-13 
—O0« 9027 1224838612720-13 
Oe 6615667609692496D-13 
~0 051897879932932320-13 
Oe 43215431233534220-13 
~0 0 37404422S5S3280280-13 
Oe 3372302437298913C~-13 
— Oe 31 38$032787129430-13 
0 e 30 246007718S5S489D—-13 


UNUNAUNAUANNNE Pere Pp heeeeUWw 


3 
4 
5 
-5 


1200000 
000000 
-0 200000 
0200000 
-0 200000 
0.00000 
~0200000 
0.00000 
-02¢00000 
0 e00000 
-0 200000 
0e 00000 
1200000 
0e00000 
- 0200000 
0200000 
— 02 00000 
0200000 
—0 000000 
02¢00000 
~0 000000 
0200000 
—0 200000 
0200000 
1e 00000 


SS TOP 
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Oe 10000000000000000 O01 
Oe 3024600771 8594890-13 
~0 0313890 3278712943D-13 
Oe 14304214378636390~-12 
— 029027 1224838612720-13 
0 e66156676096924960-13 
—Oc« S51 8297879932932320-13 
0 04321543123353422D-13 


. 00 37404422993280280-13 


Oe 33723024372985130-13 
0 e 31 3890327271 25430- 13 
Oe 3024600771 85S5489D-13 
Oel1ND00000000C00000D 01 
0 e 3024600771 859489 0-13 
—0« 30301014222996770-12 
0 0 1430 421437863639D- 12 
—O5e 90271224838612720-15 
Oe 6615667609692496D-13 
—-0e5189787993293232D- 13 
Oe 43215431233534220-13 
~ 003740442299 3280280-13 
0033723024 372985 130-13 
~—Oe« 313890327871 29430-13 
0 «3024600771 859489 0-13 
Ose 1 0000000000000000 O1 


fy 


oa Nb oop 


me He we ms 
Ow & Wht ia) 


oe 


o 


JG 


NOHOANACANNANE 


0 


Onc u 


(a) 


a 
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*PUSSELL * s TIME =399PAGE S275 
PROGRAM ¢3 


THIS ROUTING EVALUATES Tre APPFUXIMATION Tuo THe 
KRCNECKtLR OCLTA FUNCTICN AND USeS IT Ta CALCULATE 
THE ESTIMATcD AMPLITUDE & WHEN THE =. AL AMPLITUDCS 


ASS SPCCTIFIEDe DtURLEe FAL TISTUuN Is US-cD TS FS scLVE 
THe EXACT VALUES GF TRE CFFeDIAGONAL CLEMEATS © 


RLAL*S THATASAMAT si UUT Cre UNN a DOU CR eT he TNOURK ePIeDEO IS 
Reabe5 CTFUECGOG) eC oUM sCMELX(400) 
INTEGCF OsF 


OG S 1=14460 
CTFUECT I =G 00 


INPUT DATA 


N=10 

M=3 

TN=301 

TF=1leS 
CTFUEC(S J =2 eS 
CTRUE(7)=00e38C 
CTRUECC3I=CTRVUE(S) 
C7VRVUE (1 D=ECTRUE(7) 


NN=2 ®*N41 

ONN=NN 

MV¥=2 =M41 
FIi=301415226535897 


FRINT &5 

FORMAT CEL eSX ef Qa TX et CCA) 91 EXs PCL) o/) 
CS S$ I=1leNN 

G=I-N-1 


AMAT=C eS 

Cu S64 K=1,.N 

DR=K 

THe TAH2e th LT FOKATN® CD CS TNMOR/TE )/ONN 
AMAT =DCOS( THETA) 4AMAT 


Cc 
32 
Cc 
33 30 
34 
35 99 
36 $1 
37 
36 
$e 
Q 
-10 
-S 
—& 
-7 
-3 
-5 
—4 
-3 
-2 
-1 
0 
1 
2 
3 
4 
5 
6 
7 
3 
9 
10 


SsTOP 


440 


DEL TRE=ACLe t2e*AMAT)I/STNA 


CSUM=CTFUe (L ) FCEL TF Q+CSUM 
CMPL XCITV=CSUM 

PRINT GleQoCMFLX(I)sCMPLX( I) 
FCFMATCTS8sFl0e3sCS2Gelt} 


STOP 


ENS 


NTRY 


ce) 


0.084 
0 e336 
—02e08S 
02042 
-0 e916 
~00032 
0214S 
200936 
—-0e253 
00176 
00161 
Gel76 
0 e253 
2e03f 
0014S 
~0 e032 
~02010 
0 e042 
—0 2089 
O0235¢ 
0 e084 


cc) 


Oe &4340726424789E7T-Ol 
Oe BSS7IS1IIPTESELZEIC OD 
—00€9165°0351596234C-01 
Oe 220SES F3C4424E38C-01 
~O«elO40E1CZ2E74C4210-C1 
~—O02e 31735510E0331816C-01 
Oe 1LSESTECSCSIE7310D OC 
Oe20S7ESLATIZ4EOO4SD C1 
~Je 255477704 0584983C 00 
De I P7E4ESCLIESCSECEEC CC 
—“05e16134397S736F1A8SC 00 
Oo 17€48221E€3¢¢C2086D CO 
~OeeS34777C40SE4SEBO OC 
Oe 2O037EZI47IZ4EO04SC O1 
De istESILESCSIcC 72100 CC 
00317355 10GS022181EC-Cl 
~Ce 1040810228746431C-01 
Oe 4Z20GESSIC4GE4CSETD-O!l 
~—OeS9 1655 0 351SS¢€2345-01 
Oe SES7IOLIF7ESZ3I24€C 00 
De S43807SEAI47ESCETDO-O1 


mn is iu 


JC6 


AANANNAANANNNEG 


AAAN 


°o 


ANANANDN& 
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®RUSSELL* e TIME=H3BC PFLCES=H7S 
FROSF AM O4 
OISCRETE FCURTERP TRANSFUKM ANALYSIS 


THIS FRCGRAM CAN EE USED TC CEASRATE - CICITAL 
FOURTIIR TRANSFORM FROM TIME SERTES ChTAs IY wltt 
ALSO CC & FIRST-CACER LINEAR REGRESSION ANALYSIS 
GN THE INPUT DATA TC FERCVE TRE CC-VALUE ANE ANY 
LINEAS TRENDe THE ESTIMATED FULFTEF COEFFICIENTS 


ARa THEN USEC TC COMPUTC AN ESTIMATES TIME SERIcCSe 


CI4ENSION FVALUE C2CL Ie FSAVEC EOL) oF xX( E64) 240400) 
CIMENSTEN CMAGN(450) 2 CERF ASA(C40C)2F (400 22 YFC( 201) 
09 2 1=1+400 

ACT )=040 

@(1)=G.0 


INPUT: CATA VALUES 


LINK&EG=1 

NS =41 

TOIL “606 
TF=1-0 
FNS=NS 
TN=FANS*®TDEL 
SCALE =S2e8 


ALS=AS 
NMAX=(NS=—1 97/2 
ECV=9 50° 


READ OATA CARGS ANG PRINT CATA VALUES 


READ CS eZ CF VALLE CL) 2T=H1.AS) 

FORMATC10O%*H 1402) 

FRINT 6 

FURMATC#2® oh LX et Kt se 7X eof TIME® oLSXe* FIC TIMED * /) 
Cf 19 Y=1sNS 

XOELTA=I*TCEL 

PPINT oI »XDELTAsF VALUE CI) 


OC A FIFST~CFCEF LINEAR REGRESSICK ANALYSIS TO 
REMOVE THE CC—VALLE AND AAY LIREAR TRENT 
IF cCINREG=056 Tre LINSA&R ZEGRESSION IS SKIPPEDe 


BZERO IS THE Y-INTERCEFT., El IS TRE SLCFE. ANC YEAR 


IS THE AVEF AGE OF TRE DATA VALLESe 


23 
24 
25 
26 
27 
26 
25 
30 
31 
32 
33 
34 
35 
Zé 
37 
3E 
3s 
40 
41 
42 
43 
44 
as 


i7 


19 


25 
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TF CLINRES) 22.20517 

KSUM=0 

KSUM2=9 

YKSUM=0.0 

YSIUM=0.20 

Do 19 K=1e85 

KSULM=KSUM+ K 

KS IM2=KSUMS4(KA%e ) 
YKSUY=YK SU MEK *FVALLE CK) 
YSUM=YSUM+FVALLE (CK) 

XE AR=(TCEL *KSUM)/FENS J 
YRAR=YSUM/ERS 
XSUM2Z=(TCEL**2) 8K SUMS 

XY SUMZTIDEL* YKSLM d 

B= CXVSUM— XBAR FVGAR FFAS) SOXSUM2-F NSH CXEAPER2) ) 
SZERG=YEAR—-C1aXxXEAR 

PFINT 25 
FOURMATC®l®,LOxXs *EZERC % ep LOXs PH 1% oe lL axe *YEAF Fy) 
FEINTeSZERCeCl1 oY EAE 

NQ 27 K=1leNS 

YF CK D=EZERCH+EL OK PTCEL 

FSCAVE (CK) =F VALLE K) 

FV4L UECK JEFVALUE CK )-YF (CK) 


THe TIME SERIES HAS ACw SEEN CCRRECTEL FOR ANY 
LINEAR TREND AND THE CC-VALLE HAS SEEN KEMOVELCS 
TRE MCCIFIEC TIME SERIES IS NGw PRINTEDe 


AAANnRAN 


NOnw 


34 


COMPUTE COMPLEX FOCURTER AMFLITLOF S, 


CONT INVYE 

FRINT 29 

FURMATC ELS Lex eK pg 7X eo TIME* pL SK e* FE TIME V9 Ss) 
Co 39 [=iehS 

TI=I-AMAx—1 

XOUL TAHITI * TOE 

CFINT STIX CELTASFVALUE{( 1) 


CALL ULNSFTMCF VALLE cA sEefhLS OLCV ) 


PP INT 24 
FCRMATCPTEL® s4X— IN 4% 4 2Xe TOCOC—-VALUE §'s SXo *CMAG M5 10X 


Le CPHASE® 5 UXe* FFE C—riZ® 63X09" CMEC-CE PY ) 


NUMAX=(NLS—-1)72 

DC 35 [=hehLMAX 

xI=I 
CMAG=C(ACTI¥ACTIEGCI RECT) B*C OSS Ze 
CM=CMAG/SSCALE 

CMAGDIE=200 FALCGIOICH) 


E CUATICN( 12065) 


79 
80 


20 


21 


aw 


NANDNNANNnNRNAAWW 


p+ 
Ww 


40 


4> 
46 


ANN 
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ATEST=ECI) SACI) 
ATESTHATEST#ATEST 

ATESTH=ATEST#®00S 

AT=1009 e-ATEST 

LF(AT) 20020021 sd 
CEHASE=.E8E6EE8E 

69 T3 22 

CONTINGE 

BA=-3C1 SACI) 

CFKASE=ATAN( EA) 

CPHA SE =CDHASE457 626577 

FNLS=NLS 

FREQ=XI/(TCEL*ENLS) 

CMAGNC I) =CMAG 

CEHASK (1 )=CPRSASE 

PPINT 30eI el OvVec MAC ps CERASE ,FREC,CWAGFR 
FOQMATCI 6s 46 82030F 1062) 


THE TOTAL PCWEF IN TRE SIGNAL ITS NCw CETERMINED ey 
SUMMING THE SQULAFES OF THE CCWFLEX AMFLITUCES. TRE 
CUOMSLEX AMPLITUCES AR® SCALEC BY TRE SQUARE ROOT 
OF THIS TOTAL FCWEFE SE THAT ALL AWPLITUTES CAN S& 
REFERENCED TO TRE TOTAL FOWER« THIS NORMALIZATION 
WILL HELF IN SELECTING THE #CST STATISTICALLY 
SIGNIFICANT SPECTPAL CCMFOCNENTS« 


FRINT 43 

FORMATCE LY eGX oe 9 Ne 2X eo PMAGNITLUOE * pSX 5 PPEASE * 9 BX 
le *FREGUENCY's4Xe"*CE SELATIVE TO*s 
2e4AXe*UNITY TCTAL FOWEFPS) 

TP #R=0.20 

CC 40 J=l ys ALMAX 

TP af =TPRR+ CMAGAC J) &CWMAGR( SS) 
TPWR=TPwF**2005 

DG 45 J=leNLWAX 

CMAGN( J) =SCVAGR (JD / TEAR 

XJ=J 

FCEQ=XJI/ CTLEL ENS ) 
COI=2Oe*ALTGIIO(CHMAEN(U)) 

PRINT 460J+CMAGN(J )+CEEFASN(J) eFFEC eCOE 
FGRMAT(16*3ELSe3 eF1lV0e2) 


COMPUTE TRE ESTIMATED TIME SERIES, FOLATICN(12066) 


NS=4*NS—-2 
TOEL=TCcL/4G 
N¥AX=(AS—-1)/2 
TFTA 
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GS TALL FSRSCEM oe Ast sOOVeNSe TOEL OTE ) 
S$? FRINT 30 
3é eye) FOQMATC 81 oS LIEXe Ke 7X e*TIME* oe 11X 
Le®*SSTIMATEC F(TIME)*7) Y 
$$ DC 100 =i ehS 
100 LI=I-NMAX-1 
101 XDELTA=II*TOEL 


bore 100 PRINT oI IT sXDEL TA ot XI) 
103 Sol ST op 


104 END 
Cc LINE SPECTRUM SLBRCLTI AE 
Cc 
Cc THIS SUERGUTINE GcNERATES TERE FCURIER CCEFFICIENTS 
Cc A ANC & FCF NLS SAMPLES CF THe FUACTICNs FVALUE. 
€ 

10S SULBRCLTINE LASFTMCFVALUE eAsEseALSSECV ) 

106 DIAGCNSTON FVALLE € 201) 2A( 400) .2(400) 

107 NLMAX=(NLS-17072 

109 PT =301415526 

10°93 XNNENLS 

110 OC V=06 a 

111 DO 15 J=1sNLS 

l1l2 15 DCV=DCV4FvaLUE( J) 

113 OC V=OCV/ XNA 

114 D3 16 J=1eNLS 

11s lo FVALUE CJ SFVALULE CJ )-OCV 

116 00 10 A=1s8LMAX 

117 ASUM=06 

116 BSUM=0. 

11S XN=N 

120 00 5 J=H=1leNus 

tat JJ=J~NUNMAX~—1 

122 XJJ=HISJ 

123 AF G= 2 cO4F LT @XNOXII/S XNA 

124 ASUM=CTS (ARG) BFVALLE (5) 4aSUM 

125 Ss BSUM=SINCARG) ¥F VALLEC J) 4+8SUé 

126 ANI =2 eO#ASUMSKNN 

127 10 BIN} =2e040SUN/XAA 

128 CO 17 J=1,NLS 

12g 17 FVALUE (CJ) H=FYALUE TID FCCV 

130 SPE TURN 


131 ENO 


132 
13s 
134 
135 
ise 
137 
13é 
13S 
140 
14] 
142 
143 
144 
145 
146 
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€ FLUFILE SERIES SUERGUT IWE 
c 
Cc TRIS SUBROUTINE GENERATES SAMPLES CF THE DATA 
c FUNCTICNs FXC(I)« USING The FOURIER COEFFICIENTS,» 
Cc ACT)». BOT). AND OC-VALUEs CCv»e ANC THE NUNMSEF OF 
c DESIREC SAMFLES»s NSe TCEL IS TRE SAMPLING PEFIGD 
é ANS TF IS TRE FCUFIFF FERICC. 
Cc 
SUBFCUTINE FSRS CFXsAs Gel CVeNSe TOEL o TF) 
DIMENSION £X(E04) 64(600) E6400) 
NMAX=(NS—-1)72 
71 =301415926 
XNS=NE 
CO 10 I=1,AS 
K=I~NWM4X-] 
XK SK 
FxXK=0O6 
CS 5 N=1eNMEX 
XN=N 
AF G=02e03P T#XNAXK*XTDEL ISTE 
5 FXK=FXK#A( KA) *CCS CARG FEC N) OS IN(ARG) 
10 FXCI)= FXK + DEV 
RETURN 
SENTRY 


Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 


x 


OMNOW FUND ~~ 


10 


Tie 


0 e60CO000EFE 
Oel250000E 
0 el1EO00000E 
0 e2400000EF 
Oe300C000E 
0 «36CCOOOE 
0e4200000E 
Oe4SCOCCOF 
0 eS400C0CE 
Oe60C0000E 
0 e66CO000E 
0 e7Z200000E 
Oe 7B00CCOE 
0 &400000E 
0e9000000t 
CoS6CCO00E 
0 el OZ0000F 
GelO&0000E 
9 el L40000E 
OselZ200000E 
0«126CC00E 
0 e1 S20000E 
0el Z80000E 
02e1440000E 
OelLSOOO0OOE 
Col SGQ200E 
0 ei 6 20000E 
O0el680000E 
Oe1l74C3COE 
Oe180000UE 
Oel&S0000E 
0e1S20000E 
0 el 9B0000E 
0e2040000E 
0 e2Z100000F 
Oe21600C0F 
0«e2220C00E 
O e228000CE 
0e2340000E 
0 e2400000E 
0 «2460000E 


92 
03 
03 


O04 


04 


C4 


04 
04 
04 


04 
04 
04 


C4 
04 
04 
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FCTIME) 


—NVeEIZSG2SSSE 
~—U00e 1643E00E 
—CecECOECIeE 
Oe S2F1I7SSE 
—CeezSCz00ce 
—Ce1Q10ECCE 


CeE4OSQ00E 
Ce 1 SS62SCOE 
OeeS0@EC0E 
Ce 2205500E 
Colbii13scce 


—~OecC83000F 
—~Ce€Exe?2coo0e 
—Ceoltt1SNOCE 


0eF5S65000E 
Col4Gl7SCOtL 
Oel360C00E 
Oe 1SS77CCE 
0. 147@50CE 
OeSIEOQOOOLE 
C2 1425000€ 


—CeIS24001E 


Oe¢SO07O001E 
Ce. ?75310CCE 
OeclLIVSOOE 
O29 7030CE 
Oe eS€lESSE 
OelSS4100E 
Ge 7478C0CE 
Ce4120000t 
Oesd2u7CO0E 
O.1f240CCE 
G-224E000E 
Oe237SS0CE 
Cee414100EF 
Col 943700E 
O-11S5C0CE. 
0e«S912000E 
061357S00E 
0e€14S5100E 
0e2344S00E 


700883431 7E O02 


K 


-20 
-1¢ 
-18& 
~17 
-16 
-15 
-14 
-~13 
~-l2 
—-11 
-10 
~S 


&BZERO 


TIME 


—Jel2CuUCdoEe 
—“O0el149000t 
—0.1080000E 
—JVel AD 20000F 
-~0 -9S600000F 
—~C0eSOCOCOCOE 
-0 e8400000E 
-Ceo78CO0COF 
—-0.72000CO0E 
~0 e6500000E 
—OedCCOOGOE 
—0 e5400000E 
“3 e4BOO000E 
—0e42000CUE 
—G 3000000F 
—0e3N00000€F 
—-Ge24COG00F 
-Oel SQQ000E 
—Oel2coogo0o0r 
—0e600000GE 
Ge0000000E 
Ce6000000EF 
0 «1 20V000F 
Oe lL BQD0O00E 
O0+«2Z4C00COE 
O «30000008 
Ceo 36COD00FE 
0242000007 
0«4800000E 
Oe S4000CO0E 
0 e600000GE 
0e6600000E 
0e7200000E 
0«7Z00000E 
Cod4COCOUE 
0eS90COCO0E 
0e9600000€F 
Ce LOZOOCOE 
0 .1000G00¢ 
0011409000€ 
O6.120000GE 
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FCT IME) 


Ce leseSSéEGE 
-OeCZISIGEE 
—~Ceo22zececdreBe 
—-CecéC4é€Sée 
—Oe24#ZEGI24E 
—Cell CEC7ZE 

Cell SeS2cer 

Oecd ll EVO4STE 

Ceo c€SEC aE 
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*RUSSELL * e TIME=30 sPAGES=75 
PROGRAM OS 


FAST FOURIER TRANSFORM SPECTRAL DENSITY 


THIS PROGRAM ILLUSTPATES A SIMPLE EXAMPLE FOR THE 
USE OF THE FAST FOURIER TRANSFORMe SUBROUTINE FFAST 
CAN BE USED IN OTHER PROGRAMS WHERE HIGH SPEED 
PROCZSSSING IS NEEDED. THE SUBROUTINE CONTAINS A 
ZERO FILLING ROUTINE TO ACCOMODATE ANY LENGTH DATA 
AND ANY DESIRED DISPLAY RESOLUTION. 


DIMENSION A(1024),.B(01024) »CMAG( 1024) 

DIMENSION FMAGC 1024) ,FPHASE( 1024) »sFR( 1024) 
OIMENSION 301024), SNAME( 5) .XX(1024) 

DIMENSION X(1024).XMDC( 1024) »XSV(1024)eXTIME( 1024) 


READ INPUT DATA -- THE TIME SERIESs X(1I). THE 
NUMBER OF DATA SAMPLES+ NS» THE SAMPLING INTERVAL. 
DELTA-Ts THE DESIRED FFT RESOLUTION GPTIONs»s NRESL> 
THE DC-REMOVAL OPTION» NOCs ANDO THE PRINT OPTIONS» 
NPT1 e+e NPT2s NPT3s NPT4. 


READ (5 65eEND=S01) NPT1s+NPT2sNPT 3eNPT45N0C 
+ »NRESLeNS sXTDELT « SNAME 
FORMATC 4I1¢16s215:E1 Be 7511Xs 5A4G) 


***e* USER SPECIFIED READ FORMAT *#** 


READ(Sse1LOCXCI) » f=15NS) 
FORMAT(E14. 7) 


EEK EKREEEERK ESE SEE KKK EKREKA HE SCKRKARKAKARE 


PRINT INPUT DATA 


PRINT 12SsNPT1NPT2eNPT3eNPT45NDC>. 
+ NRESLeNSsXTDELT 

FORMAT(C 8 1* si 7Xs"FAST FOURIER TRANSFORM®,. 
+1Xs*SOCECTRAL ANALYSIS!///510Xs°PTOPT* 23X> 
+ *NDC® 3 3X0 *NRESLE 8 3Xs"*NS* ep 6X0 * TOELT—-SECe*//210X> 
+41155X0e1102172E13.7///) 


COMPUTE THE AVERAGE (GF DC-VALUE)»s THE MEAN-SGUARE, 
THE VARIANCE. AND THE STANDARD DEVIATION. 


XNS=FLOAT(NS) 
XBAR2=0-0 


14 
15 
16 
17 
i8 
19 
20 


21 
22 
23 
24 


25 
26 


27 
28 
29 


30 
31 
32 


33 
34 
35 
36 
37 
38 
39 


454 


DO 20 I=1.NS 
DCV=DCVEX(T ) 

20 XBAR2Z=XBARZEXCIV*XCT) 
DCV=DCV/XNS 
XBAR2=X BAR2/XNS 
XVAR=(XBAR2-DCV#OCV )*XNS/CXNS-1 6) 
STDE V=X VAR¥¥0 65 


c 
Cc COMPUTE THE ZERD-MEAN DATA FUNCTION AND TRANSFORM 
Cc THE DATA TO STANDARD VARIABLES BY DIVIDING EACH 
Cc ZERO-MEAN DATA VALUE SY THE STANDARO DEVIATIONe 
C 
DO 25 L=1eNS 
XTIMECT J=FLOATC 1-1) *XTOELT 
XMDC CI) =X€1)-DCV 
25 XSVCTE)=XMOCCIISSTOEV 
Cc 


PRINT 30+SNAME<s DCV + XBAR2¢XVAR»sSTDEV 
30 FORMATC® *4///7910Xe*INPUT DATA FUNCTION STATISTICS:* 
$e2XeS5A4Gs//98Xs TAVERAGE § 6 SX *MEAN—-SQUARE ® SX 
+*VARIANCE* eSXs "STANDARD DEVIATIGN® s//eSXsE1L265S 
$a SXe Eb 2%e Se 3X vEll eS ae 3KXsE1L!2eSa///) 


Cc 
IFCNPT1 eEQe 0) GO TO 50 
PRINT 35 
35 FORMATS 867X088 s3Xe" INPUT DATA FUNCTIGN® 3X 
+,*ZERO MEAN® ¢S5Xs * STANDARD VARIASLES® 2/) 
09 40 I=1sNS 
40 PRINT 4551-eX(1I) »sXMDC(T) »XSV(I) 
45 FORMATC® 2180236187) 
Cc 
c SELECT THE DESIRED FORM OF THE DATA FUNCTION 
c FOR FFT PROCESSEINGe 
Cc 
So NOC=NOC-1 
. IFONDC) 75255265 
ss D9 60 I[=1seNS 
60 XCI2 =XMOCCI) 
GO TO 7S 
65 DO 79 I=1sNS 


XCD) =xXsv(i) 


fo) 


THE INPUT TIME SERIES HAS BEEN PREPROCESSED INTO 
A CONVIENENT FORM. THE FFT COEFFICIENTS ARE NOW 
COMPUTED. 


FILL THE FFT ARRAY WITH TIME SERIES DATA AND 
COMPUTE THE REAL AND IMAGINARY COMPGNENTS OF THE 
COMPLEX FOURIER AMPLETUDE CCEFFICIENTS. 


NNANNNANANAN 


40 
41 
42 
43 
44 
45 
‘46 
47 


48 
49 
SO 


S1 
$2 
53 
S4 
$s 
S6 
S7 


58 
s9 
60 


61 
62 
63 


64 
65 
66 


67 
68 
69 


7S 


80 


Annan 


5 


AANN®D 


fo) 


NONNnD 


5 


ANNO 


100 
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CONTINUE 

DG 80 I=1.NS 

ACI) =xXC( 1) 

B¢{i1)=0.0 

K=0 ; 

CALL FFASTCAsB8sNSesNRESL eK) 
N=2*% (K¢NRESL ) 

K MAX=2*4(K+NRESL-1 ) 


CONVERT THE REAL ANDO IMAGINARY COMPONENTS INTO 
MASNITUDE AND PHASE COMPONENTS. 


DO 85 T=1s5N 
FMAG(TI)=CACTIO A010 +8019%*801)) F025 
FPHASE( 1 )=57 e29577*ATAN2Z( B01) cACT)) 


COMPUTE THE ESTIMATED DISCRETE FOURIER AMPLITUDE 
SPECTRUM BY SCALING. 


XN=FLGATON) 
DO 9O I[=1 «KMAX 
L=I-1 
XL=FLOAT(L) 
XXCI)=FMAGCI) *XTOELT 
CMAG(I)=FMAGCI)/ XN 
FRCOEI )=XtL/(XN®XTDELT ) 


COMPJTE THE ESTIMATED POWER SPECTRAL DENSITY 
FUNCTIONe 


0G 9S I=1+sKMAX 
L=I-1 
SCI) =CMAGCE) €CMAG( I) *XTDELT#XN 


PRINT RESULTS 


IF(NPT2 e£Q- O ) GOTO 115 

PRINT 100 
FORMATC*1*,10Xs"FFT COMPLEX GQUTPUT* 6//29Xo*N* sXe 
+E MAG % 9 7Xo fF PHASE * 16X08 ACN) *28Xo. *BIN) 8 of) 

DO 105 I=1.N 

PRINT 110eIsFMAGCI) sFPHASECT IACI) BCI) 
FORMAT(C*® *,19,4EF12.3) 


IFCNPT3 eEQ-6 OF GO TO 135 
PRINT 120 
FORMATCTL* s1LOXs* AMPLITUDE SPECTRAL DENSITY®* »//>¢ 
+9X ob %s6Xs*FREQUENCY—-HZ* SX 
+e"*MAG XX(WI-V/HZ* 5 7Xo*CMAG—DFT* 47) 


70 
71 
72 
73 


74 
7S 
76 


77 
78 
79 
8c 
81 


82 
83 
84 


85 


86 


125 PRINT 130sL eoFRC( I) »XXCI) sCMAG(I) 
130 FORMAT(C*® ',59,3615e27) 
NN 
135 IFC(NPT4 .EQe OF SO TO 499 
PRINT 140 
140 FORMATC*2*s10Xs* POWER SPECTRAL DENSITY's//7s9Xe°L* 
+7Xs*FREQUENCY—HZ'»6Xs"MAG SCI) —-W/HZ* 7/7) 
DG 145 I=1+KMAX 
L=I-1 ; 
145 PRINT 150eL+FR(C1).S¢T) 
150 FORMATC® %I1952E19.7) 
499 CONT INUE 
Cc 
Cc ***2e USER SUPPLIED PLOT ROUTINES *##¢ 
Cc 
Cc @@ INPUTS 0a 
Cc 
Cc TIME VARIABLE.» XTIMECTI)« 
Cc INPUT DATA FUNCT ION>s XU). 
c DATA FUNCTION WITH OC-VALUE REMOVEDs XMDC(I).- 
Cc STANDARD VARIABLESs XSV(Tde 
Cc 
Cc 22 OUTPUTS ae 
Cc . 
c FREQUENCY VARIABLE IN HERTZs FR(I)- 
Cc AMPLITUDE SPECTRAL DENSITY» XX(T de 
Cc POWER SPECTRAL DENSITY» S(IT)- 
Cc 
GS TO 4 
Sol STOP 
END 
Cc 
Cc FAST FOURIER TRANSFORM SUBROUTINE 
Cc 
a SUBROUTINE FFAST (AsBe NSeNRESL 9K ) 
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DO 125 £=15KMAX 
L=I-1 


DIMENSION A(1024).8(1024) 


TEST THE TOTAL NUMBER OF POINTS AND IF NS IS NeE. 
TO 2*#*K, FILL THE REMAINING TIME SERIES ARRAY» ACT) 
WITH ZEROS. IF ADDITIONAL DISPLAY RESOLUTION IS 
DESIRED» ZERO-FILLING CAN BE IMPLEMENTED BY 
SELECTING THE FESOLUTION CONSTANT». NRESLe 


DETERMINE THE POWER-OF-TWO GRDER» Ke 


ANANNAANANA 


87 
88 
89 
90 
91 
92 


93 


a 
~ 


95 
96 


97 

98 

99 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
1:9 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 


Aan 


AanNN 


10 


20 


30 
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ZNS=FLOAT(NS) 


XK=ALOG(ZNS)/ALOG( 220) 


K=INTCXK) 

N=2**K 
TFONS«GTeN) K=K+1 
N=2**(K+NRESL) 


ZERO—FILi-e 


NSP1=NS+1 
DO 5 1I=NSF1.6N 
2(1)=0.0 


COMPUTE THE FFT ON 
DECOMPSSITION WITH 


MR=9 

NN=N-1 

DO 20 M=1.NN 

L=N 

L=t72 

IFCMR4+L eGTeNN) GO 
MR=MCOCUMR.~LI +L 
IFCMReLEeM) GO TG 
TR=A CM+13 
A(M+1)=ACMR41) 
A(MR41)=TR 

TI=8 (M+1) 
B(MR+1)=TI 

CGNT TINUE 

L=1 

LTFCL eGEeN) RETURN 
ISTEP=24i_ 
EL=FLOAT(L) 

DO 4 M=1elL 


N DATA POINTS USING TIME 
INPUT BIT REVERSAL e 


Z=32e1415S926*FLOATC1—-M)/EL 


wR=COS(Z) 
WI=SIN(Z) 

DO 4 I=M,N,ISTEP 
J=I+b 


TF R=WR*¥A( J)-WIFBCJ) 
TI=WR*¥B(J)+WI*A( J) 


A(JV =ACI)-TR 
B(J)=B8C1)-TI 
ACI) =A(1)4TR 
BCI) =8C1)4TI 
L=ISTEP 

GO TO 30 

END 
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FAST FOURIER TRANSFORM SPECTRAL ANALYSIS 


PTOPT NOC NRESL NS TDELT-SECe 


1111 9 0 1S 0-3000000E G0 


INPUT DATA FUNCTION STATISTICS: EXP(-TISIN(T) 


AVERAGE MEAN—SQUARE VARIANCE STANDARD OEVIATION 
O0.11129E 00 0e27753E-01 0-16443E-Cl 0.1Z2823E 00 
T INPUT DATA FUNCTION ZERO MEAN STANDARD VARIASLES 
1 O-OQ090000E OC -Oelli2o22e 00 -0.86791S57E O00 
2 062189268E 00 Ceol076345E 00 0e8393911E 00 
3 0e3098824E 00 0-1985902E 00 O-1S48711E O01 
4 0631 84770E oO O6.2071847E 00 Oel61S5S736E 01 
Ss 0e«2807248E 00 0.1694326E 66 eiS21324E O11 
6 O«e2225712E 00 OellL2790E OC 0e8678120E 60 
7 0-16097S9E OO 0-49633E3E-01 0e3874592E OC 
8 OeLOS7OSSE CO -0e5586743E-02 —-0243568385-01 
9 0-6127660E-01 -0.-SO001S64E-01 -0.3900484E 900 
10 0e2872230E-01 —~0« 8256990E-01 —0-643923EE 00 
11 Je 7O26002E-02 -0.1042662E 00 -O0.8131231E 900 
12 -0.58181 992-02 -O0.1l171104E O00 -0.2132890E 00 
13 —0.1209130E-01 —-O061233835E 0G —0e9622099E 00 
14 —061392170E-01 -021252139E 00 ~O0.9764843E 00 
1s —021306980E- 01 —~9e1243620E 00 —-0.969840SE 00 


FFT COMPLEX OUTPUT 


N FMAG FPHASE ACN) BC(N) 

1 Oe167E O1 O.000E 00 Oel67E O01 O.O000E 00 
2 Ool27E OL -O06836E 02 O«l141E CO -0-126E O01 
3 O.473E 00 -O0.134E 05 -0.326E 00 -0.343E 00 
4 O6226F 00 -O01S53E 03 -Oe201E 00 -0.103E 00 
Ss Ool40E 00 -O00254E 03 ~0.135E CO -0.379E-01 
6 GelOSE 00 -04172E 03 -0.104E 00 -0.144E-01 
7 O-887E-01 -O.177E 03 ~-0.886E-01 —0e491E-02 


ine 


NOwnPuwnnw © 


NOnewnn © 


0.819E-01 -O00179E 
0. 819E-01 Qel 79E 
0. 887E-01 Oel77E 
02.105E 00 O01 72E 
0.140E 00 Ol 64E 
0.226E 00 0e153E 
Oe473E 06 Oel34E 
Oe.127E 01 6e836E 


FREQUENCY-HZ 


0 e0000000E 
0-2083333E 
0«416666EE 
026249999E 
0 «8333333E 
0201041 666E 
0e1249999E 
0-21458333E 


00 
oo 
00 
00 
00 
O12 
Oi 
O12 
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O03 -0.819E-01 
O03 -0.799E-01 
03 -0.819&-O1 
O35 -0.8865£-01 
O03 -0.104E 00 
O03 -0.6135E 00 
03 -0.201E 00 
03 -0.326E 00 
O02 041415 00 


AMPLITUDE SPECTRAL DENSITY 


MAG XX€CW)-V/HZ 


OeSOO081IS7TE 00 
0.3807855E 00 
0.1419684E 09 
0-«6781936=-01 
0-4204089E-01 
023137103E-01 
0«2661949E-01 
O0«2456124E-01 


POWER SPECTRAL DENSITY 


FREQUENCY~-HZ 


0. 0000000E 00 
Oe 2083333E 00 
0.4166666E 00 
O0e624S99SE 00 


9. 8333333E O00. 


Oe 1041666E O01 
02.1249999E O}1 
0.1458333E O1 


MAG SCI) -W/HZ 


0e5225342E-01 
0.3020782E-01 
0 e419896S5E-02 
06SS582227E-03 
C- 36821 58E-03 
022050294E-03 
0061476245E-03 
001256780E-03 


-0e130E-02 


OeDO0DE 00 
Ge130E-02 
0«491E-02 
0-144E-0O1 
0«379E~-01 
OelO03E 00 
O0e343E 00 
Oe1l26E O1 


CMAG-DFT 


0e«1043366E 00 
0.79330335-01 
0«2957676E-01 
001412904E-01 
08 758S19E-02 
0 «6535631E-O02 
0 e5S45728E-02 
02511 692S5E-02 


NANANNANNE 


an OOO ANANFANnNA 


pet 
n 


anna 


20 
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*RUSSELL * s TIME=30ePAGES=75 
PROGRAM 06 
ESTIMATION OF SINGLE-SINEWAVE PARAMETERS 


THIS PROGRAM COMPUTES THE BEST FIT OF A SINGLE- 
FREQUENCY SINEWAVE TG AN INPUT TIME SERIES WHEN 
THE SINEWAVE PERIGD IS KNOWN OR ESTIMATED. 


DIMENSTIGN X( 1090) sFS(1000) eXR( 1060) »XMDC(1000) 
DIMENSION SNAMECS) oXTIME(1000) 


READ INPUT PARAMETERS 


READCSe5 s2ND=S61) NSeTZEROsTOELT » SNAME 
FORMAT(IS5sSXe2E1Ge62 7X sSA4) 


eee USER SPECIFIED READ FORMAT *€**#*% 


READ(S,1090X(1)+IT=1»NS) 
FORMATCE 14.7) 


KEK AKKKKSEKSSRHERKERKE KEK SESE KHEKEKES ETE 
PRINT IN@UT PARAMETERS 


PRINT 15 eNSsTZEROsTOELT 
FORMATC® 1% s9Xs*SSTIMATION OF SINGLE-SINEWAVE * + 
+*PARAMETERS!///s11Xs "INPUT PARAMETERS: *//311X 
to tNS!? e5Xe*TZERO-SECe*% s7Xs* TDELT-SECefMS/ 

+1149 251727) 


COMPUTE THE AVERAGEs MEAN-SGUARE;< VARIANCEs AND 
STANDARD DEVIATION OF THE INPUT TIME SERIESe 


XNS=NS 

DCV=0.0 

XBAR2=020 

DG 29 E=1sNS 

DC V=DCVtEXCI) 

XBAR2 =XBAR2+XC1) €XC1) 

OC V=DCV/SXNS 

XBAR2=XBAR2/ XNS 

XV AR=( XB AR2— DC V# OCV ) FXNS/ CXNS- 1.6 ) 
ST DE V=XVAR¥*0 0S 


COMOUTE THE ZERO-MEAN DATA FUNCTION 


19 
20 


21 
22 


23 
24 


25 
26 
27 


28 
29 
30 
31 
32 
33 
34 
35 
36 
37 


38 
39 


40 
41 


42 
43 


es 


Naan 


31 


nanAN AnaAnA 


AnaAN 
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DO 25 t=1,NS 
XMOCCI)=X(3)-DOCV 


PRINT THE. INPUT TIME SERIES STATISTICSs INPUT 
TIME SERIES», AND ZERO-MEAN DATA FUNCTIONe 


PRINT 30+ SNAME s0CV sX}BARZsXVAR»STDEV 


FORMATC® §/77510X-'I 


NPUT 


DATA FUNCTION STATISTICS:* 


+ 22Xs SAQ/S 8X PAV ERAGE 9 4X5 *MEAN~ SQUARE * 4 4X 


+o* VARTANCE® s3Xe"STANDARD DEVIATIUN® //3X+4E14eS///) 


PRINT 31 
FORMATS "55Xe%T * 64X 


00 35 I=1.NS 
PRINT 3602+X€1)+XMDC 
FORMATCI7sE18e¢4eE21. 


2** INPUT TIME SERIES®* 
+s4Xs"ZERO-MEAN TIME SERIES'S) 


Ct) 
4) 


COMPUTE EQUATIONS (15615) AND €13-216) USING THE 


ZERO MEAN DATA FUNCTIE 


N= (NS-1) /2 
Pl=3-1415926 
ZEN=C2e*PI*TOELT I/TZ 
At PHA=0.0 

BETA=0-0 

00 40 I=1,.NS 

L=I-N-1 

AL =L 


AL PHA=AL PHA+XMOC (1) *COS(XL¥*¥ZED) 


BE TA=BETA+XMOC (I )*Si 


AL PHA=AL PHA/ XNS 
BE TA=~-BE TA/XNS 


ONe 


ERO 


N(XL*ZED) 


COMPUTE THE MAGITUDE AND PHASE OF THE COMPLEX 
AMPLITUDE COEFFICIENT GIVEN BY EQUATIONS 


C€230e17) AND (136138). 


CMAG= CAL PHA# ALPHAtSBETA®*BETAI¥¥*065 
CPHAS E=S 702957 7# ATAN2(BETA+ ALPHA ) 


COMPUTE THE MAGNITUDE AND PHASE OF A SINGLE- 
FREQUENCY SINEWAVE WHICH IS A BEST FIT. 


DMAG=24*#CMAG 
DPHAS E=CPHASE-270 2 


PRINT THE FREQUENCY, 
SINEWAVE. 


MAGNITUDE.» 


ANO PHASE OF THE 


45 


61 
62 


4&5 


Nana 


so 


60 


aan 
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FREQ=1.-/TZERO 

PRINT 45 eFREQ »DMAGs DPHASE 
FORMATC*®1%s7Xe*BEST—-FIT SINEWAVE2®9//.7%X 

+e * FREQUENCY ® 9 7Xe ° DMAG®* » 8X » *"DPHASE-DEG*// 5 4Xe 


#3E1424//7/) 


COMPUTE THE "RESIDUE" TIME SERIES AND 
ITS STATISTICS. 


DPHAS E=2 e #PI* DPHASE/360 « 

00 50 I=1.NS 

L=I-N-1 

XL=L 
XR61}=XMOCC I) -DMAGESEN( XL*#ZED4¢DPHASE ) 
OCV=0 e0 

XBAR2=0-0 

DG 60 I=1,NS 

OCV=DCVFXR(T) 

XBAR2=X BARStXRCI) €XROI) 

DC V=0CV/XNS 

XBAR2=XBAR2/XNS 

XVAR= (XSAR2-O0C VB OCV ) ¥XNS/CXNS—1 2 2 
ST DEV=XVAR¥#0-5 


PRINT THE “RESIOUE™ TIME SERIES SYATISTICS. 


PRINT 70,0CV»XBAR2.XVAR»STDEV 


FORMAT(* %s10Xe*RESIOUE TIME SERIES STATISTICS: *§77/ 


+28Xe* AVERAGE® 6 4X oe *MEAN-SQUARE ® 94.X0 * VAR EANCE® 
+e3Xe" STANDARD DEVIATIONS //3X14E14e5/7// ) 


PRINT THE ZERO-MEAN DATA FUNCTION®s BEST FIT 
SINEWAVE. AND RESIOUE TIME SERIES. 


PRINT 75 
FORMAT(C*® "/// 614Xs° REFERENCE® 0e26X%s* RESIDUES » 


tGOXe*T*eGXe* TIME SERTIES® « 7Xs"SINEWAVE * 
+eoe8SXe* TIME SERIES*//) 

DO 80 i=1sNS 

L=I—N-1 

XL=t 

KT IME (1 )=FULOATCI-1) *TOELT 
FSCE)=COMAG#SINCXL €ZED+DPHASE ) 

PRINT 90eteXMOCCII.FSCI) »XRCI} 

FORMAT (172351724) 


*¢4¢ USER SUPPLIED PLOT ROUTINES ***#+% 


@2 INPUTS aa 


c TIME VARIABLE» XTIME(CT)« 
c INPUT DATA FUNCTIONe XCI)-~ 
Cc ZERO-MEAN DATA FUNCTION, XMOCCI). 
c 
¢c aD OUTPUTS dad 
c 
c SENEWAVE FUNCTION, FSCI). 
Cc *-RESTOVE" TIME SERIES» XRCIT). 
c 
Cc RES SSEKCHSESCHKKHKKSK SEEKS EK ERE SCHERER KEES 
c 
72 Go To 4 
73 S01 ST99 
74 END 
ESTIMATION OF SINGLE-SINEWAVE PARAMETERS 
INPUT PARAMETERS: 
NS TZERO- SEC. TOELT-SEC. 
41 O64926101E O3 O«E000000E 02 
INPUT DATA FUNCTION STATISTICS: HER A 7/29/75 LINRESG 
AVERAGE ME AN—- SQUARE VARTANCE STANDARD DEVIATION 


Oe 72944E-04 O«149046E O05 0615276E 05 Oel2360E 03 


T INPUT TIME SERIES ZERO-MEAN TIME SERIES 
1 001626E 02 0016265 62 
2 -0.9235E 02 ~0.9235E 02 
3 -002222E 03 ~0e2222E 03 
4 -9.2805E 03 -0e280SE 03 
S -062426E 03 —-0e2426£ 02 
6 ~0e6166E 02 -0.6166E 02 
7 OolL1S3E 03 O0e.1153E 03 
8 0.2187E 03 , 0e2187E 03 
9 0.2553E 03 0-.26S58E 03 


10 0e«2273E 03 
11 OelOS7E 03 
12 —-9e3037E 02 
13 —-0e8642E 02 
14 —9e6305E 02 
15 Oe45561F 02 
16 OeLO16E OF 
17 O68773E O02 
18 0e1373E 03 
19 OeS8I2Z0E 02 
20 ~0e2320E O02 
ei ~0.6858E 02 
22 —921264E 03 
23 -0e5421E 02 
24 —-063212E 02 
25 0e9816E 02 
26 Oel733E 03 
27 0.12435E 03 
28 Oe1S35E O02 
29 -9-7344E 02 
30 —-0211S52E 03 
31 -~9e1019E 03 
32 -0.2029E 02 
33 Oe4375E O02 
34 0-4894E 02 
35 00e442S5E 02 
36 -Oe1l095E 02 
37 -0-9798E 02 
38 —-04e1225E 03 
39 ~Q0.9404E 02 
40 —Oc«e2304E 02 
41 0.83838E 02 

BEST-FIT SINEWAVE: 

FREQUENCY DMAG 
022030E-02 
AVERAGE ME AN—- SGUARE 


Oe1IS7SSE 00 


Oe«l285E 03 


0+-66434E 04 
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0e2273E 
O«.1097E 
-0.3937E 
—-0.8642E 
~02e630S5E 
Oe4S61E 
O«1LOLG6E 
0e8773E 
Oel373E 
0«8120E 
-0-e2320E 
-0-6558E 
—Uel264E 
-O0eS421E 
~0e3212E 
0e«9816E 
Oe 1 733E 
061243E 
9-1535E 
—0.7344E 
—0«e«1152E 
-OelO0L9E 
—02e2029E 
0«4375E 
0e4894E 
0e4425E 
-0-2109S5E 
-0-9798E 
—-0e1225E 
—0-9404E 
—-02e2304E 
0«8838E 


DPHASE-DEG 


~0.21053E 03 


RESIDUE TIME SERIES STATISTICS: 


VARIANCE 


0-63995E 04 


STANDARD DEVIATION 


0e82S520E 02 


REFERENCE 
TIME SERIES 


Je 1626E 
—0292355 
“0222225 
—0.22805E 
04624265 
—-0 261 66F 

QellS3E 

Oe2187E 

Ve 26585 

002273 

O«1L097E 
—Oe30R%7E 
—02e8642E 
—-9- 6305 

0e4S561E 

OeLO16E 

0.8773E 

Jel3Z7ZE 

0.8120E 
—-022320E 
~0-e68S8E 


" —00eS421E 


—06«32i12E 
0.9816E 
Oe1L733E 
9e1243E 
Oe1S35E 

—Oe 7344E 

—~Oe1152E 

~OelLOLOE 

-0.22029E 
0243 75E 
0048 94E 
004425E 

—-O0elOG5E 

-0.9798E 

—O0e1225E 

—~0294045 

—002306E 
0.38838E 


02 
02 
oS 
o3 
03 
02 
03 
03 
03 
03 
03 
e2 
02 
02 
02 
03 
02 
03 
02 
02 
02 
03 
02 


nn 
Ww 


02 
03 
03 
02 
92 
03 
03 
02 
02 
02 
02 
32 
02 
03 
02 
02 
02 
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SINEWAVE 


O.lL273E 
Ue 7988E 
—-O0.1209E 
-O.9731E 
~0.1283E 
~-O0.8770E 
Oe1774E 
0.9026 
0-.1284E 
0 .9496E 
0.8S551E 
—0.8262E 
~O«l277E 
—0O.1016E 
—0+.1882E 
0.74465 
061262E 
CelO76E 
0«2897E 
—0 e6581E 
-O .1239E 
~O04e1129E 
—-C -3893E 
6 s6SS73E 
0«1208E 
O«1li74E 
9-4864E 
0 «4729E 
~OeLLESE 
—O061212E 
—0-eS803E 
0.3754E 
Oe LL22E 
O.1245E 
Oe 570SE 
-0.2755E 
—0 -1068E 
—-Gel26S5E 
~-Ce7563E 
O0e1738E 
Oe1007E 


O3 
02 
02 
02 
03 
02 
01 
02 
Q3 
02 
01 
02 
03 
03 
02 
02 
03 
03 
02 
02 
03 


03 . 


02 
02 
03 
G3 
02 
02 
03 
03 
02 
02 
Q3 
03 
02 
O02 
o3 
03 
02 
02 
03 


RESISVE 


TIME SERIES 


—OcelLIOE 
—~9el722E 
-Oe2101E 
—0201822F 
~Ocll44e 
0e2604E 
O«e11S55E 
961284E 
Oe1ZS74&E 
Ol 3S26E 
0-1012E 
0.5225E 
0-41 30E 
0-38S5S5E 
0-6443E 
062716E 
—0.3848E 
0«2969E 
0eS224E 
06.4%260E 
0e-SS30E 
~Oe«l3S48E 
—0.1S28E 
~0.-8885E 
—0.22S59E 
0.5S84E 
0. 7562E 
0-5264E 
O0-«4340E 
0e6065E 
—~0e4383E 
-0-eS783E 
—~0.6843E 
-~Oe7531E 
—0e2280E 
O- 1660E 
928809E 
Oe 3960E 
~Oel1841E 
~064042E 
-~00e1232E 


03 
03 
03 
03 
03 
02 
03 
03 
03 
03 
03 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
O1 
02 
02 
02 
02 
02 
02 
01 
o1 
02 
02 
02 


PWN = 


51) 


~OO OND 


> 


l2 
13 


14 


is 


16 
17 
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° 
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®"RUSSELL* »TIME=30+PAGES=75 
PROSRAM 07 
MAXIMUM ENTROPY SPECTRAL ANALYSIS 


THIS PROGRAM COMPUTES THE MAXIMUM ENTROPY SPECTRAL 
ESTIMATE FOR A TIME SERIESe THE PROGRAM WILL DOA 
FIRST ORDER LINEAR SEGRESSICN ANALYSISe IF DESIRED» 
AND ALSO CONVERT THE DATA TG STANDARD VARIABLES. 


DIMENSIGN AK(100)2FR( 1000) 

DIMENSION S(1000)sSPLCT(1000) sSNAME(S) 
DIMENSION X(1006)»XMDC(1000) »«XNORM(1000) 
DIMENSION XSV(1000) »XTIMEC(1000) 


READ INPUT DATA -- THE TIME SERIESs X{(1).5 THE 
NUMBER OF DATA SAMPLES» NS» THE MAXIMUM NUMBER GF 
CGEFFICIENTS TO 3E CALCULATEDs MMs THE SAMPLING 
INTERVAL « DELTA-Ts» AND THE DESIRED RESGLUTION IN 
THE MEM SPECTRAL ESTIMATEs DELTA-Fe SNAME IS A USER 
DEFINED LABEL FOR THE INPUT DATA SET. LINREG IS 

A PROCESSING OPTION. NPT is NPT2.s NPT32 NPT4e ARE 
PRINT OPTIGNS. 


READ (5S sEND=S01) NPT1eNPT2eNPT3sNPT4 
+eL INREGsNSsMMeXTDELT »FDELT » SNAME 
FORMAT (411,16 s215.E13e3sFi50e325A4G) 
N=NS 
XN=FLOAT(N) 
LFLFDELT) SQ1507 
FOEL T=025/(FLOAT (NS) ®XTOELT) 
CONT INUE 


**x2* USER SPECIFIED READ FORMAT **** 
READ INPUT DATA FUNCTION’ 
READS», 10). ECXCI) s¥=1.N) 


FORMAT (1 0X0F 1402) 
RRKKSEKEEKHKCEEKEKESEKKEKKEE KKK HEE 


PRINT INPUT DATA. 


PRINT 15 
FORMAT(41%s17Xs"MAXIMUI ENTROPY SPECTRAL ANALYSIS‘. 


t4/729Xe* PTOPT* 92X% yp ®LINREG® 6 3X0 *NS*95Xs TMM 8s 
+3Xe*DELTA-Ts SEC*.3X-e*DELTA-F. HZ*/7) 


PRINT 20sNPTI sNPT2eNPT3eoNPT4sLINREGoNSs MMeXTDELT sFDELT 


FORMAT ® %09X041154Xel1le3XeISel7 sElLS 07 cElL607) 


18 
19 
20 
21 
22 


23 
24 
25 
26 
27 


28 
29 
30 
31 

32 
33 
34 
35 
36 
37 
38 
39 
40 


41 
42 
43 
a4 
4S 
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DO A FIRST ORDER LINEAR REGRESSION ANALYSIS TO 
REMOVE ANY LINEAR TREND AND COMPUTE THE 
AVERAGE VALUE. 


K SUM=0 

K SUM2=0 
XKSUM=0.0 
XSUM=0.0 
XOC=06 


COMPUTE THE AVERAGE VALUE. 


00 30 K=1eN 
XDC=XDC+XCK) 
XDC=XDC/XN 

DC 35 K=1.N 

XMDC (CK) =XOK)~xOC 


COMPUTE LINEAR REGRESSION. 


DO 40 K=1eN 

K SUM=KSUMtK 

K SUM2=K SUM2+(K*%2 } 
XK SUM=XK SUM+FLOAT (K)*X(K) 

X SUM=XSUM#X(K) 
XT BAR=XTDELT#EFLG ATI KSUM) /XN 

X BAR=XSUM/XN 
XT SUM2=X TDOELT # #2 = FLOAT KSUM2) 
XTSUM=XTDELT*XKSUM 
BX1=(XTSUM-XT BAR EXBAR¥XN) SOXTSUM2—-XN¥ (XTBAR*¥ 20) ) 
BZERO=XBAR-BX1*XTBAR 

DO 45 K=1eN 

XNORM(K ) =X 0K )-BZERO-8X1*K*XTDELT 


THE TIME SERIES HAS NOW SEEN CORRECTED FOR ANY 
LINEAR TREND ANO THE OC-VALUE HAS SEEN REMOVED. 


COMPUTE THE STATISTICS FOR THE VARIOUS FORMS OF 
THE DATA FUNCTION. ALSOs TRANSFORM THE DATA INTC 
STANDARD VARIABLES BY DIVIDING EACH ZERO-MEAN DATA 
VALUE BY THE STANDARD DEVIATION. 


PZERO1=0-0 

PZERO2=0-0 

PZERO3=0-0 

DO SS I=1eN 
PZEROI=PZERD14X( 1X1) 


46 
47 
48 
49 
So 
52 
S2 
S3 
54 
SS 
56 
S57 


58 
$9 


60 
61 


62 
63 
64 


65 
66 
67 


68 
69 
70 
7i 
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73 
74 
7s 


76 
77 
78 
793 


SS 


60 


65 


70 


7S 
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PZERO 2=P ZERO 2+XNORM( 1) *XNORM( I) 
PZERO3=P ZEROZS+(X( I) -XDC) *(X(1)-xDC) 
PT=PZEROI/XN 
PZERO1I=PZERO1/( XN-1.-) 

PZERO2=PZERG2/ (XN-1e) 
PZERG3=PZERO3/(XN-1.2) 

STDE V2=PZERO2**0.5 
STDE V3=PZERO3** 0.5 

D2 60 I=15N 

XI=I 

XTIMECL J=XI*¥XIDELT 
XSVCTI)=XNORMCI) /ST DEV] 


PRINT 65+sSNAME+s XDC o PT »sPZEROZ es STDEV3 

FORMAT(*® %e///510Xs* INPUT DATA FUNCTION STATISTICS:', 
#2X oSA40//48Xs SAVERAGE® 5X 9 *MEAN—-SQUARE® 5X5 
+ ©VARIANCE* 54Xs *STANDARD DEVIATION® 6//745X% 5 E1265 
He3XevEl20 5s 3X E12 eS 3XsElL20Se//S/) 


PRINT 70s8ZERGsBXi sPZERDZ.STDEV2 
FORMAT(*® ".9Xs"*DATA FUNCTION STATISTICS AFTER 's/4010X» 
+*L INEAR REGRESSION: * 3//s8Xs *BZERO® o11Xe*Bl*e10Xy 
+* VARIANCE 54X96 "STANDARD DEVIATION® 6//0S5XsE1205 
#9 3X 9E 120 593K 9 E12 6 Se BX cE1LZ2eSe/S//) 


IF(NPT1 eEQe 0) GO TO 90 

PRINT 75 
FORMATS 57X—5°T*,3Xe* INPUT DATA FUNCTION® 94X, 
+*LINREG® 97X»s*STANDARD VARIASLES*,/) 

99 80 I=1eN 

PRINT 8521sXCI)»XNORM(I) XSV(I) 
FORMAT(C® *,1853E18.7) 


IFCLINREG «EQ. 3) GO TG 115 
LINREG=LINREG-1 
PZERO=PZERO1 

TF(LINREG) 125995,105 


DG 100 I=1.N 
X€1)=XMOC{I) 
PZERC=P ZERG3 
GO TS 125 


0G 110 I=1.N 
XCL)2=XNORMCT) 
P ZERO=P ZERO] 
GO TO 125 
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DO 120 I=1,N 
XCI)=XSVCI) 
PZERO=1. 
CONTINUE 


THE INPUT TIME SERIES HAS BEEN PREPROCESSED INTO A 


CONVENIENT FORM. THE MEM AUTOREGRESSIVE COEFFI- 


CIENTS ARE NOW COMPUTED. INPUT VARIABLES ARE X(I); 


PZEROwNsPT»MMsXTDELT*eAND FDELT. 


CALL ATCREG(X »sPZEROsNeo PT sMMeXTCELTsFDELT. 
+PK sAK eK s NPT2 »NPT3) 


THE AUTOREGRESSIVE COEFFICIENTSs AK{T)» HAVE NOW 
BEEN ESTIMATED USING A LEAST-SQUARE ERROR 


CRITERIGNe THESE COEFFICIENTS ARE NOW USED IN THE 
MEM SPECTRAL ESTIMATOR TO OBTAIN AN ESTIMATE OF THE 


POWER SPECTRAL DENSITY FUNCTION. 


CALL MEMSPMCNS 4K» XTOELT sFDELT PK sAKs Se NMAXs 
+SBIGeIMAX»NPT4) 


COMPUTE SPECTRAL AMPLITUDES IN DB AND 
FREQUENCIES FOR PLOTTINGs 


N¥P 1=NMAX+1 
DO 499 Y=1eNMP1 

L=I-1 

FRCI )=FLOAT(L)*FDELT 

SPLOT( I }=10.€AL0G10(S(1)/SB1G) 
STEST=SPLOT (1 )4+406 

IFCSTEST) 498 2498,499 


SPLOT(I )=-40- 
CONT INVE 


#**e USER SUPPLIED PLOT ROUTINES #4#¢*% 
@a INPUTS @@ 


TIME VARIABLE, XTIME( I). 
INPUT DATA FUNCTION» XCI)-6 

DATA FUNCTION WITH DC-VALUE REMOVEDs, XMDC(I).- 

DATA FUNCTION AFTER LINEAR REGRESSION: XNORM(I)« 
STANDARD VARIABLES +sXSV(I de 


aa OUTPUTS 3a 
FREQUENCY VARIABLE IN HERTZ, FRC I) 6 


POWER SPECTRAL DENSITY FUNCTIONe SCI). 
RELATIVE PSD IN DSs SPLOT(I). 


95 
96 
97 


983 


99 
100 
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103 
104 
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106 
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E81 
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114 
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116 
117 
118 
119 


120 
121 
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GOTO 4 
stop 
END 


THIS SUBROUTINE ESTIMATES THE AUTOREGRESSIVE 
CGEFFICIENTS USING A MAXIMUM ENTROPY CRITERION 
AND A MINIMUM LEAST SQUARE ERROR, 


S UBR OUT INE ATOREG(XsPZEROsNoPT s MMeXTDELT eFDELT >» 
+PK sAK eK es NPT2 NPT 3) 


DIMENSIGN AAC100)sAK( 100) ¢AKSAVE( 100) 
OIMENSI ON B102000) »82(1000)sP( 1000) .X(1000) 


INTEGER T,.TP1 


COMPUTE THE STARTING VALUES» B1(T); B20T). AKO1)> 
AND P(€1) FOR THE RECURSIVE EQUATIONS. 


K=1 
B1€1)=xX (1) 
NMi=N-1 
B2CNM1)=X(N) 
D0 S T=2.NM1 
Si¢T) =x (TT) 
B2CT—1)=XCT) 
XNOM=G6 
XDEN=0 6 
NMK=N—-K 
DO 10 T=1.sNMK 
XNOM=XNGM+B1 (7) *3B2(T) 
XDEN= XDEN#B 207) *81(T)452(7T)*B2CT) 
AK(1 )=2 « *XNOM/X DEN 
AKSAVE (C12 )Y=AK(C13 
PC1)J=PZERO*( 1 «-AK( 1) #AK(1)) 
PRINT 15S 
FORMATC*1%s7X. "MEM STARTING VALUES 5 * 6S /3EX9"K* es 8Xo 
+ PALL) 8 s11Xe*PC1)%s8Xe*P(0)*s5X»s "TOTAL POWER®,«/) 
PRINT 20eK+AK(1) sP(1) »PZERO,PT 
FORMATCI7sE1 707236126 3///) 


a: 


COMPUTE AK(T) VALUES USING RECURSIVE EQUATIONS 


K=K+1 
NMK=N—K_ 
KM1=K-1 
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125 DO 3G T=1.KMi 

126 30 AAT) =AK(T) 

127 vG 35 T=1+NMK 

128 TPI=T+] 

129 BlCTI=61(0T)—-AACKM1) 4#32(T) 

130 35 B2CT I=B2CTP1I-AA(KM1) #381 (TP ) 
131 XNCM=0-6 

i32 XDEN=06 

133 DO 40 T=1.+NMK 

134 ANOM=XNOM4+810T) *32(T) 

135 40 XDEN=XDEN+81(07T) *3107T) 43207) #3520T) 
136 AK(K )=2 e ®*XNOM/XOEN 

137 AKSAVECKI=AKCK) 

13s PCK)=PCKM1)*(1.-AK(K) €AK(K)) 
139 OCG 45 T=1.KM1 

146 KMT=K-T 

141 AK(T ) =AA(T)—-AK(K) ®AACKMT ) 


44 

Cc 

c TG INHIBIT THE PRINTING OF ALL IMMEDIATE FILTER 
c CALCULATIGNS AND COEFFICIENTS» CHANGE STATEMENTS 
Cc 452 5028 AND 51 TO COMMENTS AND CHANGE THE RANGE 
Cc OF THE OG-LOOP FROM STATEMENT 45 TO STATEMENT 44, 
Cc 


142 45 PRINT 46sKsTsAK(T) 

143 46 FORMATC® *,218eE18.7) 

144 50 PRINT 51 

165 Si FORMAT(//7/) 

146 IFCMM—K 952255925 

147 S52 PRINT 53 

1438 5S IF {NPT2.2EQe0) GO TO 71 

149 $3 FORMATC® *,10Xe¢#*k ERROR IN ATORES #44 ,//7) 

1590 PRINT 60 

151 60 FORMAT(*® "s7Xs*SUMMARY OF ITERATIVE RESULTS*s//,% 


+5X o* INDEX® 3X0 "COEFFICIENT * »6X-2*ERRGR POWER*, 
+77 97TXo*K%eBXe "AKC K) 89 1 3Xe*PIK) *2/} 


152 PK=P(K) 

153 OS 65 I=16K 

154 65 PRINT 7OseIsAKSAVE(CI) »P(T) 

155 70 FORMATCI8s3E17e7//) 

156 71 PRINT 7SsKsPC(K) 

1S7 7S FORMATC*® 1%, GXs*AUTOREGRESSIVE ESTIMATION SUMMARY!, 


+7/798Xe LENGTH: K="' eI 3e3Xs"ERROR POWERs P(K) =", 
#E1 307 2///28EXs "ORDER * o 3X 

+*AUTOREGRESSIVE COEFFICIENTS* »// > 

+BX o®T*eOXs*AKIT) ' 3/7) 


158 IF(NPT3 eE9- 0) SC TO 90 
159 00 80 T=1.MM 
160 _ 80 PRINT 85sTsAK{T) 


161 85 FORMATC® *.18sE1867) 


162 
163 


164 


165 


166 
167 
168 
169 
170 


171 
172 
173 
174 
175 
176 
177 
178 
179 
180 
181 
182 
183 


184 
185 
186 
187 
18383 
189 


190 


191 
192 
193 
194 
195 
196 
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RETURN 
END 


THIS SUBROUTINE EVALUATES THE MEM SPECTRUM USING 
THE CALCULATED COEFFICIENTS, AK(T). AND THE 
SPECIFIED FREQUENCY INCREMENT» DELTA-Fe THE ROUTINE 
CALCULATES THE NYQUIST FREQUENCY AND LIMITS THE 
FREQUENCY RANGE ACCORDINGLY. THIS ELIMINATES THE 
POSSIBILITY OF COMPUTING ALIASED SPECTRA. 


SUBROUTINE MEMSPMUNS 0K sXTDELT sFDELT ePKy AK» S eNMAX 
#sS3IG sIMAXsNPT4) 


OIMENSTON AKC100)sS5(1000) 


PI1=321415926 
NMAX=INT (16/( 206 #XTDELT#FDELT) ) 
NMP 1=NMAX+1 
PRINT 1 
FORMAT(*1*,12Xe*MEM PCWER SPECTRAL DENSITY* elxs 
+*ESTIMATIGN® s///) 
DO 10 I=2,NMP1 
L=I-1 
XL=FLOATIL) | 
PHI=26*PI*#XL*FDELT#XTOELT 
ALPHA=0.0 
BETA=0.0 
DG S J=1eK 
ALPHA=AK (J) *COS ( J# PHI )+ALPHA 
BETA=AK£J)*SIN( J#PHI)+8ETA 
D=(€ 1 e~ ALPHA )* (1 «.-ALPHA)+BETA*BETA 
S(CI)=PK4#XTDELT/D 
PRINT 14 
FORMAT(® "55Xe*THE LARGEST SPECTRAL COMPONENT IS:*, 
+77 o7Xo*L "34Xs "FREQUENCY 5 7Xs "AMPLITUDE *5/) 
S$B81G=0.0 
O00 1S I=2 sNMP1 
XS=S(CI ) 
IFCXS e¢GT.SBIG)ISBIG=XxsS 
1S=0 
IS=I1S+1 
STEST=S(IS)-SBIG 
IF(STEST) 20221521 
I MAX=1S=1 
F RE Q= I MAX*F DELT 
PRINT 25s IMAXsFREQ.SBIG 
FORMATO" %el7eEl2e4sE1Ge4s///) 
PS=0.0 
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197 DO 30 I=2sNMAX 

198 30 PS=PStSCI)*FOELT 

199 PS=PS4S(1)*FDEL T/2-6+S(NMP1)*FDELT/2.6 

200 PS=2-4*PS 

201 PRINT 31sKePS 

202 31 FGRMATC£® ",5Xs "THE TOTAL POWER IN THE ESTIMATED®, 


+1lXs*SPECTRUM IS® 57/7.6Xe* LENGTH s K=* 91 395X, 
+*POQWER=" sEILZe%e//SS//) 


c 
203 IFCNPT4 .EGe. 9) GO TC 45 
204 PRINT 35 
205 35 FORMATC*® ©,6X0"L *s6Xe *FREQ-HZ'sSXs "NORMALIZED" 
+3Xe"TOTAL POWER! 3X. * RELATIVE e/s27Xs" SPECTRAL", 
+1 9X0" AMPLITUDE °4/527Xe *DENSITY® 9 7Xs "DB" 01 3X0 (OB Ye /) 
206 DQ 40 I=1.NMP1 
207 i=I-1 
208 FREQ=L*FDELT 
209 PS1=S(1)*FDELT 
210 PS1=10.*ALOG10(7S1) 
211 PS2=S(1)/SBIG 
212 PS$2=10.¢ALOG1 0( 9S2) 
213 40 PRINT 4lelsFREQ+S( 1) ePS1sPS2 
214 41 FORMATC* %517¢2E1303eFllelsFidel) 
215 45 RETURN 
216 END 
MAXIMUM ENTROPY SPECTRAL ANALYSIS 
PTOPT LINREG NS MM DELTA-Ts SEC DELTA-F, HZ 
1111 3 41 15 Qc6000000E 02 0.2500000E-04 


INPUT DATA FUNCTION STATISTICS: HER A 7/29/75 
AVERAGE MEAN— SQUARE VAR TANCE STANDARD DEVIATION 


02e82960E 02 OeSLIO2ZE OS 0.2482SE 05 Oe1LS7S6E O03 


DATA FUNCTION STATISTICS AFTER 
CINEAR REGRESSIONS: 
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VARIANCE STANDARD DEVIATION 


8ZERO e1 


—-O0. 88H2E 02 O.1SS95SE 02 Del Se76E O05 0.12360E 03 


T INPUT DATA FUNCTION LINREG STANDARD VARIABLES 
I ~0«63S2S99E 02 0e1625516E O62 Oel3LS16S5E 900 
2 ~O4.1043800E O3 -9.9235213E 02 ~067471983E 90 
3 —O«2860801E 03 ~0e222209SE 02 ~Sel1l797842E 01 
4 -023361799E O3 ~O0.2804666F O02 ~Ce225Si85E Gi 
5 —0.2902000E 0O3 —042426439E 02 ~0.1963172E O1 
6 —Oe1lQ1L0600E O03 -90e6166125E 02 —0e4988860E 00 
7 Oe8405000E 02 0.115291SE 03 069327948E 00 
8 0-1956200E 03 Oe2187041E 03 Oe 1769481E 01 
9 0.2508800E 03 0«2658066E 03 O-.2150576E 01 
10 0.2205500EF O35 0e227219SE 03 Ce183S186E 01 
11 OelLLLIZSOOE O3 0e1CE7423E 03 0eS878S80E 00 
le —-O0.20829000EF 02 -0.3037497E 02 —-0.2457564E 00 
i3 —0e6872000E 02 —9«8642226E 02 —Ce6992212E 90 
14 ~OQe3719000E C2 ~0e6304SSSE 02 ~0e5101184E 00 
1s O-.7963000EFE C2 0-4561317E 02 0e36904S50E 00 
16 0e1437900E 03 Oe«1G1L6159E 03 0e8221490E 00 
17 O«.L380600E 03 0O-«8772861E 02 0-«7097906E 06 
i8 O.1957700E O03 021372812E 03 Oell10708E 01 
1g9 021478500E 03 Oe8120407E O62 0e6570022E CO 
20 O.5150001F oO2 -002320322E 02 ~O4e187731SE 00 
21 O0e1438000E O02 ~O-e«68S8051E 02 ~0e«5548680E 00 
22 “O4e3S24001E 02 ~O0.1263578E 03 —0.1022329E 01 
23 O64507001F£ 02 —-0e«S420S03E Gz ~Oe428SS96E 00 
24 Oe7S31000E O02 —0.e3212231E 02 —0«2598937E 60 
25 Oe2137S00E 03 O0«981E025E 02 O«.7941904E 00 
26 Oc2970300E C3 O0e1732830E 03 Oo1401990E O01 
27 Oe2561S99E 03 0el2425S56E 03 O.1005322E 01 
28 Oe1SS4100E 03 0e6153485S9E 02 061241816E 00 
29 O0«7478000E 02 ~“O6e7343871E 02 -0.594174S5E 00 
30 Oe«4120000EF O02 ~O9e1LSI760E 03 —0-931860SE 00 
31 0e6267000E Ge —-0e1088E€33E 03 ~0e824150SE 00 
32 OelS24000E O03 ~0«202905S3E 02 -O0-.16416S6E 00 
33 O0.2246000E 03 0e&37S220E 02 0-3539882E 00 
34 Oe2379S00E 03 0e4894458E 02 0-395998S5E 00 
35 Oe2614100EFE O03 064424756E 02 Oe«3579961E 00 
36 0-1943700E O3 ~OelL0948971E 02 —0.8859134E-01 
37 O211SSOOOE O3 -~0-9797690E 02 —-Oe.7927070E 00 
38 0-9912000€ 02 —-O.1225141E 03 ~0.9912317E -00 
39 Oe13S7500E O03 —-0.9404160EF 02 —Ge7608674E 00 
&c Ge2149100E 03 —9.2303882E 02 —C.1864014E O00 
41 Oe 3SZ44900E 03 0-8838379E 02 O«71S0GiSE O06 
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MEM STARTING VALUES: 


Al(@1) P(1) P(0) TOTAL POWER 


0. 7703549= 00 0e407E 00 DelLO0O0E O01 Oe311€E oS 


2 i 0e1426599F 01 
3 1 061273918 01 
3 2 -0.5961846E 00 
4 1 0-1286666E 01 
4 2 -0e5537803E 00 
4 3 —-O« 2698378E 00 
Ss 1 001269886E 01 
5 2 —0.4S01221E CO 
Ss 3 —0201391941E 00 
Ss 4 —0¢2324146E 00 
6 1 061255294E 01 
6 2 —0e47S7471E 00 
6 3 —-0.1305848E 920 
6 4 —0e2021002E 00 
6 Ss O-.1573693E 00 


Oe1249800E 01 
—0e4897244E 0G 
~001126345E 00 
—021905018E 00 

0-.199624S5E5 00 
—-0+4964290E-01 
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O0.1277999E 01 
~O0.SO0S¢8S57E 00 
-~0.4925478E-01 
-O.2S5098SiE 00 

0-1638636E 00 
—GezZ0S51279E 00 

0-4856238E 00 


Oe1214496E 01 
—-0e40835S53E 00 
—-029026274E~-01 
-0e218210S5E 00 

Oe1136636E 00 
—-O4.2149794E 00 

Oe3845207E 00 
~0.6188025E-01 


0.11965949SE O01 


. 700e4137836E 00 


-02S655117E-01 
-0e2370692E 00 

0e1236346E 00 
-0023461216E 00 

0e3766007E 00 
-009770262E-01 
-0-9347177E-01 


O21195743E 01 
~02e4150690E 00 
-0.5789482E~-01 
—-O4e2318900E 00 

0.1204148E 90 
—0e2324213E 00 

Ge373Z3404E O00 
-~0-9848028E-01 
—-0-9916228E-01 
~0-7126266E-01 


12 
12 
12 
12 
l2 
12 
le 
l2 
12 
12 
l2 


14 
14 
14 
14 
i¢ 
14 
14 
14 
14 
14 
14 
14 
14 
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Oe119SS73E 01 
—-0«e4159462—E 006 
-—025911542=E-01 
—022331021£ 00 

Oe1250103E 00 
—0e2352822E 00 

0e374%8226E 00 
~02e1013346E 00 
-0-4S987491E-G61 
—~O6. 76371 79E-01 

0.9661391E-03 


Oe1195349E 01 
~—G.4159285E 00 
-0.6050623E-01 
-0.2349269E 00 

0e1231648E 00 
-042284S63E 00 

0e3705379E O00 
~0-9905803E-01 
—-O-1041199EF 90 
~0-¢7744831E-01 
~0« 66086 76E-02 

029463485E-02 


001198529E 01 
—-0e4175817E 00 
-0-5935176E-O1 
-062213914E 00 

0-1413535E 00 
-0e2111518E 00 

02630S58087E 90 
-0-5914903E-01 
—00e12563S5E 00 
-0+3641000E-01 

0-e3951150E-02 

068212191E-01 
—00 2270263E 00 


O0ol184735E O01 
—9023996552E 00 


15 
15 
is 
15 
15 
1S 
1S 


a 
a 


1S 
1S 
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a 
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~0.6S83S589E-01 
—00e2217042E 900 
002 442283E 00 
—0e2012320E 00 
0-3104789E 900 
—04683329493F-01 
~0e1089634E 00 
—-0.47S7093E-01 
02e2144169E-01 


12 0«8680815£-901 
13 —0.194CSS1E 00 
14 0« 800S691E-01 


SUMMARY OF ITERATIVE RESULTS 


INDEX 


K 


ODNOANFWN 


COEFF ICTENT 
AK(K) 


O«7703549E 00 
—~0e8518724E 00° 
—Oe.1792291E 00 

0e«7112604E-01 

0e2359126E 90 

0 e618S063E-01 

0.8881 867E-01 
—-Oe31 749590E 00 
-O 2090011 7E 00 
-0.8772367E-01 
~0e1 375243E-01 
-0.1230915E-01 
—-0.1821105E-0!1 

Cel T46899E 0G 

0.7895 762E-01 


ERROR POWER 


P(K) 


0e4065S534E 
O9e1115230E 
OelO79406E 
0.1073945E 
OolO14174E 
Ov.LOLO0294E 
0.1002324E 


00 
00 
a0 
00 
00 
00 
0° 


0-9012860E-01 
0e8652300E-01 


0.858S715E- 


01 


0-8S84088E-O1 


0085827 82E- 
0e8579934E- 


01 
o1 


0e8318102E-01 
0.826624CE-01 
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AUTOREGRESSIVE ESTIMATION SUMMARY 
LENGTHs K= 15 ERROR POWERs P(K) =0.38266240E-0i 


ORDER AUTOREGRESSIVE COEFFICIENTS 


T AK(T) 

1 0011 84735E C1 
2 ~02e3996S62E 00 
3 —9.-65 &3589E-01 
4 ~0e2217042E 00 
s 021442283E 00 
& ~02e2012320E 00 
7 0e31904789E 900 
8 -0-8329493E-01 
9 —0-e10839634E 00 
10 —-0.47S57093E-01 
11 9221441 69E-0O1 
12 9-86 80815E-01 
13 —-Oel9S05S1E 00 
14 0«8005691E-01 
is 0-78S5762E-01 


MEM POWER SPECTRAL. DENSITY ESTIMATION 


THE LARGEST SPECTRAL COMPONENT [S35 
L F RE QUE NCY AMPLITUDE 


VT? 041 925E-02 Oe42Z210E 04 


THE TOTAL POWER IN THE ESTIMATED SPECTRUM IS 


LENGTHse K= 15 POWER=0.9988377E 00 


OMNANPFWNHREO 


FREQ-HZ 


OeOG0E OO 
0 .2SSE-64 
0 eS O0E-04 
O.7SO0E-04 
0 ei OOE-03 
001 25E-03 
001 SOE-03 
001 75E-03 
0 e200E-03 
0 e225E-03 
0 e2S0E-03 
0 «27S5E-03 
0 «3 COE-03 
C e325E-03 
0 e3S0E-03 
0e375E-03 
0 e400E-03 
0 e425E-03 
0e450E-03 
0 04 75E-03 
0.S00E-03 
0 e525E-03 


0 eSSOE-03. 


005 75E-03 
0 e600E—-03 
0 e625E-03 
0 e65G6E~-05 
0 e67SE-03 
0 «7 00E-03 
0 .725E-03 
0e750E-03 
007 75E-03 
0 e8COE-03 
0 e825E-03 
0.850E-063 
0 0&8 75E—03 
0.900E-03 
0 2925E~-03 
0 «950E-03 
029 75E~-03 
0 01 OOE-02 
021 02E-02 
0.10SE-02 
0 «1 08E-02 
0 0110E-—02 
0 e112E-02 


NORMAL IZED 
SPECTRAL 


DENS 


0e287E 
Oe287E 
0e286E 
0e28SE 
OCe284E 
O0e282E 
Ceo280E 
Oe278E 
Oe277E 
Oe27SE 
Oe274E 
Oe273E 
Oe273E 
Oo274E 
Oe276E 
Oe279E 
0e283E 
0e288E 
Oe 29S5E 
0e304E 
Oe31SE 
0e328E 
Oe 344E 
Oe 363E 
0e387E 
0e415E 
0e4469E 
0e490E 
0eS39E 
0-S99E 
Oe672E 
Ce7G6LIE 
Ce869E 
CeolOOE 
Oel L6E 
Oe 1 3Z4E 
O.o1S4E 
Ge1 7SE 
OeI VISE 
Oe 205E 
Oe209E 
6.2203E 
Oe189E 
Ool71E 
OeISIE 
Oeil 3S3E 
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ity 


TOTAL POWER 
O08 


—3164 
—-3144 
-31.5 
—-31.5 
-31.5 
—31eS 
-31-5 
~31.6 
~31.6 
~316 
—~31.6 
~31.7 
—-31.7 
—-3148 
-31.6 
—-3146 
—-3145 
—-3164 
-31.3 
—-31.2 
—-31.0 
—-3029 
~3007 
—-30 4 
—300e1 
—-29e8 
~29e5 
—29e0e1 
—-28e7 
—28e2 
~27e7 
-27 62 
~2646 
~26e0 
—-2544 
—-24.8 
~-24e1 
~2306 
—-23042 
~2249 
-2228 
~22e9 
~2303 
—-2367 
~24e2 
~—24.8 


RELATIVE 
AMPLITUDE 
DB 


~21¢7 
—-2167 
“21.7 
—2167 
“2147 
—-21+47 
-218 
-21.-8 
—2168 
—-21468 
2149 
—-21.9 
~2169 
—2169 
-21.8 
—-2168 
2107 
21.26 
-21.-5 
-2104 
-21.3 
—2161 
-20.9 
-2066 
—2004 
-2001 
i907 
-19e3 
-1829 
—-18eS 
—-1820 
-1704 
-1669 
-1662 
-1526 
—-1520 
—-14e4 
-13-8 
—“1304 
~1301 
-13-0 
1362 
1365S 
“1309 
1404 
—-15e0 


Gel LSE-02 
OoL1L7E-02 
001 20E-02 
0.123E-02 
0.125E-62 
01 27E-02 
0.130E-02 
06132E-02 
0013S5E-02 
0.137E-02 
00140E-02 
0.142E-02 
0.14S5E-02 
0 .148E~-02 
0.150€-02 
0 6152E-G2 
01 S5SE-02 
00157E-02 
00160E£-02 
0.1 63E-02 
001 6SE-02 
0 01 67E-02 
01 70E-02 
0.1 72E~02 
0.1 7S5E-02 
0.178E-02 
0 «1 80E-02 
0.1 82E-02 
001 8S5E-02 
061 88E-02 
0.8 90E-02 
00192E-02 
001 9SE-02 
001 97E-02 
0 2 00E-02 
0.2 03E-02 
0 02 0SE-02 
0 02 07E-02 
0 e210E-02 
00212E—-02 
0 «2 15E-02 
C02 18E-02 
0 «2 20€-02 
0 «2 22E-02 
0 .225E-02 
0 62 28E-02 
0 .230E-62 
0 0233E-02 
0 «2 35E-02 
0 .237E-02 


Oel1G6E 
0-102E 
0e908E 
Oe813E 
Oe 735E 
0e«672E 
0e622E 
OeS81E 
C0eSS5SO0E 
C.S27E 
OeS10E 
0.%99E 
0.49SE 
0.%SS5E 
OeSO3SE 
OeSi7E 
O0eS38E 
0. S69E 
COe610E 
0-666E 
On 741E 
02e842E 
Ce981E 
Oelli8E 
00146E 
O-1LSDILE 
Oe 264E 
06398E 
Oe677E 
OelL3S8E 
6e338E 
Co421E 
OelL TIE 
0. 734E 
Oe38SE 
0e232E 
Oo1S4E 
Oe1 O9E 
O0e812E 
0Ce628E 
OeSOIE 
Co410E 
0e342E 
Oe291E 
Ce251E 
0e220E 
00195E 
Oe1 7S5E 
OelS8E 
0-144E 
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03 
03 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
02 
O3 
03 
03 
03 
03 
03 
04 
C4 
04 
04 
03 
03 
03 
03 
03 
02 
02 
02 
02 
G2 
02 
02 
02 
02 
02 
02 
02 


~25 04 


-25e9° 


—~26e4 
—-26.9 
—-2704 
~27e7 
—-28ei 
—28e4% 
~28.6 
-28-8 
-28e9 
~2940 
—-29e61 
—-29e1 
~29.0 
—-28-9 
~28e7 
-28e5 
~28e2 
—-27e8 
-270e3 
~26e8 
—-26e1 
-250e3 
~24 04% 
-23042 
-21438 
-2020 
-17.7 
-14.6 
~20e7 

-928 
-1347 
1724 
—-20c2 
~224 
—-24e1 
—25e6 
-26-9 
—-28-0 
-29.0 
-2909 
—-3007 
—-3144 
—-32-<0 
~3206 
~336! 
—-3346 
—~3420 
3404 


-1526 
—-1661 
~16e7 
~17el 
-1726 
-18.0 
—-18.3 
—186 
~18.8 
—-1920 
—-1962 
~-1963 
—-19e¢3 
-19¢3 
“1962 
~1961 
-1869 
~18e7 
—-1864 
-1820 
—-17-5 
-17.0 
-16.3 
~15e5 
~-1426 
—-1364 
-12<0 
-10.2 
—-729 
-4e8 
~-0e9 
00 
—-3e9 
-726 
—-1004 
—1266 
~-2464 
—-15e9 
~-17el 
—-1863 
~1962 
~-20e1 
—20¢9 
-21e6 
—22e2 
—-222e8 
—-2343 
—-2328 
~24e3 
—-24%e7 


0 e245E-62 
0 e243E£-02 
0 e245E-02 
0 0247E£-02 
0e250E-02 
0 e252E-~-02 
0 e2SSE-02 
0 «2 S8E-02 
0 e260E-02 
0 e262E-02 
0 e265E-02 
0 e268E-02 
0e270E-02 
0 e2 73E-02 
0027S5E-02 
0 e277E-02 
0 e280E-02 
0 «2 83E-02 
0 e28SE—02 
6 «2 88E-02 
02290E-02 
0 e292E-02 
0 «29SE-02 
0 e298E-02 
0 o300E-02 
0e302E-02 
0 0305E-—02 
023 08E-02 
0e210E-02 
0 e313E-02 
0031SE-02 
0031 7E-02 
0 e320E-02 
00323E-02 
0 e325E—02 
0 «328E-02 
0 e330E-02 
02332E-02 
00335E-02 
0 «338E£-02 
00340E-02 
00343E~-02 
0 034S5E-02 
00e348E-02 
0e3S50E-02 


| O0e353E-02 


0 o35SSE-02 
0e3S7E—02 
0 «360E-02 
0 e363E-02 


Oel33E 
Oel23E 
OeLISE 
OelO8E 
OeLOIE 
Ce962E 
Ceo91ISE 
0e876E 
Oe 880E 
0-8Q08E 
Oe 779E 
Ge752E 
Oe727E 
02 704E 
O0e681E 
0e659E 
O0e637E 
0e«616E 
Co S94E 
OeS72E 
CeSS0E 
0eS28E 
0eSOSE 
06483E 
O0e461E 
0e439E 
0e418E 
Oe 398E 
0e378E 
0e359E 
CeS41E 
Oe 324E 
Oe308E 
Oe294E 
Ce280E 
0-268E 
0e256E 
0-246E 
Oe237E 
00229E 
O«e221E 
Oe21SE 
0.209E 
Ge20SE 
O-201E 
0e198E 
0.1 95E 
00194E 
0«el SSE 
001 93E 


3468 
~35-6 i 
~— 3504 
-3567 
-36.0 
—-3602 
—-36e4 
—-36.6 
—-36-8 
~3669 
—-37.1 
~-37.3 
—-37e% 
—-3705 
~-3767 
-37-8 
—-38.0 
—-38el 
—38e3 
—-38e4 
~38e6 
—-3808 
~39 20 
—-3962 
—-39 64 
~39e6 
-3908 
-40.0 
45.2 
-~40.5 
~4007 
-4069 
~4l1le1 
—-4163 
—-41.5 
—~41.7 
-41.9 
—-42.1 
~42 63 
—4244 
—-4246 
~42.7 
~42-8 
~4269 
~43-0 
—~4361 
—-43e1 
~-4361 
—~43 02 
~4302 


-252e0 
2503 
~2Se7 
~2509 
~26042 
—2604 
~26 26 
~-2668 
—-27e0 
2702 
—-27e3 
-2745 
-2766 
—27¢8 
2769 
—28e1 
—28e2 
-28e3 
—28e5S 
~-28.7 
-28.28 
—29.0 
~29e2 
—29e4 
2906 
—-29-8 
~30.0 
-3002 
~3065 
~Sie7 
-3009 
“3161 
-3124 
“3146 | 
-3128 
—-3240 
“3262 
-32e3 
“3205 
—-32e7 
~32e8 
~3209 
~3320 
—-3361 
-3362 
—-33.3 
-3363 
3304 
~3304 
-3304% 


0 «3 65E-—02 
0 «368E-02 
003 70E-02 
0 2372E-02 
003 75E-02 
003 78E-02 
0 e380E-02 
0 «3836-02 
0 e385E—02 
0 03 88E-02 
0 -390E—02 
0 e392£-02 
003 95E-02 
0e397E—02 
0 04 002-02 
0 e4G2E-62 
0 e4 OSE-02 
004 07E-02 
00410E-02 
90412£-92 
00415E-02 
0 041 7E~02 
00420E~02 
0 0422E-02 
0 0425E-02 
0e427E-02 
0 e430E—02 
0 e432E-02 
00435E-02 
0 04 37E-02 
00440E-02 
00442E-02 
0 e44S5E-02 
00447E-02 
004S50£-—02 
004S2E-02 
0455E-02 
004S7E—02 
0 e460E~-02 
0.463E-02 
0046SE-02 
0.46 7E-02 
004 70EF-—02 
004 72E-02 
0047SE-02 
0 e477E-02 
0 -480E-02 
00482E-02 
0 e485E-02 
00487E-02 


0e194E 
0e196E 
Oe 1 99E 
02«202E 
0-207E 
Oe 213E 
0e220E 
0e229E 
0e239E 
0e250E 
0. 264E 
0e280E 
0. 298E 
06.319E 
Oe 343E 
0e36SE 
0e399E 
0e432E 
0. 466E 
Oe502E 
0.538E 
Oe S70E 
OeS9TE 
0e614E 
0e620E 
OoGISE 
0eS94E 
0-564E 
0eS27E 
06486E 
0e444E 
0e-403E 
0+6364E 
Oe329E 
0.298E 
Oe270E 
02 246E 
0«225E€ 
Oe206E 
C21 905 
Oe 1 76E 
00e164E 
Oe Ll SSE 
Oe l44E 
Oe 136E 
O01 29E 
Cei2SE 
Oe1L1l SE 
OoLI4E 
OoL1IOE 
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—436e1 
~43e1 
~43-0 
—-43.20 
~42 9 
—4267 
4202 
—4220 
—-4128 
—416S 
—4143 
—4120 
—-40e7 
—-46e5 
—-40-0 
—-39 07 
—3963 
—-3920 
~38e7 
—-38e65 
—-3843 
—38el 
—38el 
—-38el 
—-38.3 
~38eS 
—-38.8 
—-390e2 
—-396S 
—-4020 
—-40e4% 
—~4028 
—-4123 
—-41e7 
—~42e1 
—-42-5 
—42 69 
—-43062 
—4306 
—4309 
~44.2 
—4464 
—44.7 
~—44.9 
—45 61 
~45e3 
—-45.5 
—-4526 


—-3304 
3363 
—-33.3 
—33<2 
—-336e1 
—-33690 
—-32¢8 
~3207 
—3265 
-32e3 
—320 
—3168 
-3145 
—~31e62 
—300e9 
—-30 2S 
—-30e2 
—29 69 
~—-29e6 
—-29e2 
~28 «9 
—-28e7 
—-28e5S 
—-28 04 
—-28e3 
—28 64 
—-28e5 
—-28e7 
2940 
~—290e4% 
~-29e8 
—-300«2 
—-3006 
~Z3lel 
—31e5 
—3109 
~32e3 
~—32e7 
~33601 
-3365 
—-35328 
~34el 
—3404 
~34e7 
3409 
—35e1 
-35e3 
~35e5 
—350e7 
—-350e8 


196 
197 
198 
199 
200 
201 
202 
203 
204 
205 
206 
207 
208 
209 
210 
eit 
212 
213 
214 
218 
216 
217 
228 
219 
220 
221 
222 
223 
224 
225 
226 
227 
228 
229 
230 
231 
232 
233 
234 
235 
236 
237 
238 
235 
240 
241 
242 
243 
244 
245 


0 -490E—-02 
0«492E-02 
0 495E-02 
0 04 97E-02 
0 eSOCE-02 
0 «S02E-02 
0 eSOSE-02 
0-5 08E-02 
0 e510E-02 
0eS512E-02 
0 eS1S5E~-02 
6 eSi7E—-02 
0 «-5S20E-02 
0 eS22E-02 
0 eS25E-02 
0e527E-02 
0 eS530E-02 
0 eS32E~02 
0eS3S5E-02 
0 eS37E-02 
0 .540E-02 
0 oS42E~02 
0eS4SE-02 
0 e547E~02 
0 e550E-02 
0 eSS2E-02 


wee ww 


0 e5S7E—-02 
0e560E-02 
0 eS62E-02 
0e565E-02 
0 eS67E-02 
0 05 70E—-02 
005 73E-02 
00S 75E-02 
0 eS77E-02 
0«580E-02 
0eS82E-02 
0 eS8SE~—02 
0 eS87E-02 
0 eS90E—02 
0eS92E-02 
0-595E-02 
0 .S97E-02 
0.6 00E-02 
0 e602E-02 
0 06 OSE~02 
0e607E~02 
0 e610E—02 
06612E-02 
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O.1 O7E 
O01 O4E 
001 02E 
Oe 1 OLE 
0.993E 
0.984E 
Ce978E 
0.976E 
Ce977E 
O.981E 
0. 988E 
Ce998E 
O.101E 
0.103E 
OL 04E 
0.1 06E 
0.108E 
OelllE 
0.113€ 
Oo11SE 
Oe118E 
Ool21E 
00123E 
0012SE 
O.127E 
00129E 
Se231E 
00132E 
0.132E 
Oe I S2E 
Del 3S2E 
Oo1Z31E 
00130E 
0el29E 
Ocl27E 
0.125E 
0.123E 
0e120E 
006118E 
02116E 
Geii3SEe 
OoL1IIE 
0e109E 
Oe lL O7E 
0e105E 
02103E 
Oe1LO02ZE 
Oo101E 
Ce100E 
Ce997E 


-45.7 
-45.8 
-45.9 
—~46-.0 
—~4661 
—-4661 
—-4661 
~46el 
—-46e1 
—-46e1 
-46e1 
-46.0 
—-4549 
—-45.8 
~45.8 
-45.7 
—-45 26 
-48.S 
~-45.-3 
~45.2 
~4561 
—-4520 
~4520 
~44 69 
—- 44.6.9 
~44.8 
4448 
—~4468 
~44.8 
~44.8 
~44.9 
—44 09 
—-45.0 
—-45 61 
~-45el 
~45e2 
~45e3 
4564 
-45.-5 
~45 .6 
~45-7 
—-4547 
~45.8 
—~4569 
~45069 
~46.0 
—-%6.90 
~46490 


—-35-9 
—-36e01 
-36e1 
~36062 
~36.3 
—-36e3 
~36e3 
—-36e3 
—-36-3 
—-36e3 
—~36e3 
—-36.3 
3562 
—-3661 
—-36e61 
—-3620 
—-35-9 
~3508 
“3507 
~3566 
3565S 
3564 
-35e3 
~35e3 
—35602 
—-3561 
—35e1 
-~35e0e1 
~35.0 
-3500 
~-35.90 
-35e01 
—3Se1 
-350e1 
3502 
—-350e3 
—-3504 
3504 
-35-5 
3506 
3507 
—-3508 
3509 
-36.0 
~36.90 
—3661 
~36e2 
~3602 
—-3602 
—36e3 


0 e615E-02 
0 e618E~—02 
0 0620E-G2 
0 e622E~-02 
0 e625E-02 
0-627E-02 
0 e630E—02 
0 e632E-02 
0 e63S5E-02 
0 637E—-02 
0 .640E-02 
0 0642E-02 
0 e645£-02 
0 0647E-02 


‘0 e650E-02 


0 6652E-02 
0 .555E-02 
0 .657E-02 
0 .660E-02 
0 .662E—02 
0 .665E—02 
0e667E-02 
0 «6 70E—02 
0.672E-02 
0 6675E-02 
0 .677E—02 
0 06 80E—C2 
0 «683E—02 
0 -68SE—02 
0 .687E-02 
0«690E-02 
0 06 92E-02 
0 -695E—02 
0 .697E—02 
007 00E-02 
G e7G2E-S2 
0.7 0SE-02 
0 .707E-02 
0710-02 
06712E-02 
0 e71SE—02 
0 0717E-02 
06720E-02 
0 e722=-02 
00725E-02 
0 «728E-02 
0 e730E-02 
00732E-02 
0e735E-02 
0.737E-02 
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C.995E 
02 996E 
Oei00E 
Gel ODlE 
0e102E 
Oe L03E 
0e10S5E 
0.1 O7E 
Cell OE 
OeLIL3E 
OellZ7E 
Oe-121E 
0e126E 
OelLSIE 
0e138E 
Oe1l44E 
0e152E 
0e160E 
O.169E 
Co178E 
0e188E 
0. 198E 
02 208E 
Oe218E 
0e226E 
00 233E 
02238E 
On.241E 
Oo24IE 
0 238E 
0«233E 
0e«22S5E 
0e216E 
0e205E 
0.1 93E 


n 
CeiSié& 


Q.169E 
Oo1IS7E 
Ocl47E 
Oe136E 
Ov. 127E 
O0e1l18E 
OoL LOE 
0.1 03E 
0.969E 
0.e912E 
O«860E 
OoS14E 
Oe 772E 
0e735E 


~-~46.0 
-4620 
~46.0 
—-4620 
-45-9 
-45-9 
—-4528 
—-45-6 
—-45-5 
~45.3 
~45062 
-45.0 
~8448 
—-4446 
4444 
—-440e2 
—~44.0 
—-43.7 
—-4365 
—-43 03 
~43e1 
—4268 
—~42 06 
—-42-5 
—-42e3 
~42e2 
—~42e2 
~42e2 
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